Supplier Scorecard & Vendor Performance Evaluation

How period-by-period measurements become weighted criteria scores, a recency-weighted composite grade, and — once that grade crosses a threshold — an automatic block on future orders

1. Requirements

1.1 Functional requirements

• Compute a periodic, per-supplier performance score from a configurable set of weighted criteria, each a formula over named, independently computable variables.
• Represent a variable as a reference to a callable that produces one number for a supplier over a given date window — either a built-in named calculation, or an arbitrary importable code path, resolved dynamically each time it is needed.
• Represent a criteria as a formula string referencing variables by name in curly-brace placeholders, substituted with each variable’s computed value and evaluated through the platform’s constrained evaluator, then bounded to a configured maximum score.
• Enforce that a scorecard’s criteria weights sum to exactly 100, and that its standing bands cover 0–100 contiguously with no gap or overlap, before either can be saved.
• Automatically create successive scoring periods on a configured cadence (weekly, monthly, yearly), backfilling from the supplier’s own creation date to today, without duplicating a period a submitted one already covers.
• Recompute a scorecard’s composite score from every submitted period whenever a new one appears, weighting each by recency through a scorecard-level formula using the same substitution mechanism, so older periods contribute less.
• Derive a discrete standing label from the composite score by looking up which configured band it falls into, and propagate consequent flags — block or warn on new orders or quotation requests — both onto the scorecard and directly onto the linked supplier record.
• Let those propagated flags be read at order and quotation-request submission time to hard-block or soft-warn, as the companion procurement design already covers; this design is concerned only with where those flags originate.
• Ship a library of roughly twenty built-in variable calculations, mostly querying receipt, invoice, quotation-request, and quotation activity within a period’s own date window, that a new scorecard’s criteria draw on by default.

1.2 Non-functional requirements

• Formula extensibility without code changes: an administrator can add a criteria formula or variable without touching this module’s code; the arbitrary-code-path option lets a deployment wire in fully custom logic, at the cost of no code-level restriction on what that path can import.
• Idempotent period backfill: repeated invocation of the period-creation routine never duplicates a period for a date range a submitted one already covers.
• Deterministic recomputation: the same submitted periods and configuration always yield the same composite score and standing, so re-saving the scorecard is always safe.

1.3 Constraints

• Criteria weights on a scorecard, and on each period snapshotted from it, must sum to exactly 100, not merely close to it — nothing normalizes an off-total set automatically.
• Standing bands must be exhaustive and non-overlapping across 0–100; a scorecard cannot be saved with a gap or a collision.
• Formula evaluation runs through the same constrained evaluator used elsewhere in this system, applied only after every placeholder has been substituted — but nothing constrains what a variable’s own arbitrary code path can do once resolved; that boundary is enforced by write permissions, not the evaluator.
• Variable, criteria, and standing definitions are writable only by an administrative role — the arbitrary-code-path option is only as safe as that permission boundary.

2. High-Level Design

2.1 Component diagram

2.2 The period-creation walkthrough

Creating a period is a bounded, idempotent backfill rather than an on-demand action a user triggers per period:

1. Start at the supplier’s own creation date — not some arbitrary configured epoch.
2. Compute the candidate period’s end date from the configured cadence: seven days out for weekly, the calendar month’s last day for monthly, a year minus a day for yearly.
3. Check for an existing submitted period overlapping this range. If one already covers it, skip to the next candidate range without creating anything.
4. If no covering period exists, snapshot one — from the scorecard’s current criteria configuration (re-reading each formula and maximum score from its master record at this moment), extended with every variable each formula references, then immediately submitted.
5. Advance to the next candidate range and repeat from step 2 until the candidate’s start date reaches today.
6. If any period was created, re-save the parent scorecard so its composite score and standing reflect it immediately.

This backfill runs both from a daily scheduled sweep across every scorecard, and from a second entry point every time a scorecard itself is saved, so a manual configuration edit also catches up to the present without waiting for the next scheduled run.

2.3 Data flow — from a raw variable to a written-back supplier flag

3. Deep Dive

3.1 Data model

Variable (with a Scoring Variable copy attached to each Period) — a named, reusable calculation: a display label, a parameter name used inside formulas, and a path. A dotted path is dynamically imported; a bare name resolves against the built-in variable module. A Period’s Scoring Variable rows are not references to the master record — independent rows carrying their own resolved value, recomputed fresh on every save of the Period.

Criteria (with a Scoring Criteria copy on both the Scorecard and each Period) — a formula string, a maximum score, and a weight. The variables a formula needs are not declared explicitly; they are discovered by scanning the text for {name} placeholders and looking each up against the Variable list. A formula is validated at save time by substituting a dummy value for every placeholder and confirming it evaluates cleanly — a syntax check, not proof the formula is meaningful.

Standing (with a Scoring Standing copy on the Scorecard only) — a named band with a minimum and maximum grade, a display color, notification flags, and the four prevent/warn flags the gating mechanism ultimately reads. Bands are validated contiguous and non-overlapping across 0–100.

Supplier Scorecard — one per supplier: owns the criteria list, standing bands, a scorecard-level weighting formula (discounting older periods), and a period cadence. Its own status field is a plain label copied from whichever standing band the composite score falls into — not a workflow state a user sets.

Scorecard Period — a submittable snapshot of one date range: criteria and variables copied from the parent scorecard at creation time (re-reading each criteria definition’s current formula and maximum score, but carrying the scorecard’s own weight), each variable’s resolved value, each criteria’s score, and the period’s own total_score. Once submitted, a period is immutable; it contributes to the composite score until cancelled.

3.2 Algorithm — variable, criteria, score, standing

The computation is a strict pipeline, run in full every time a Scorecard Period is validated (which happens on every save, not only at creation):

1. Resolve every variable. A dotted path is imported component by component with no whitelist of permitted modules or functions; a bare name is looked up as an attribute of the built-in variable module. The resolved callable is invoked with the period itself as its only argument, producing a number.
2. Evaluate every criteria formula. Each formula’s curly-brace placeholders are substituted with the corresponding variable’s value (0.0 for any that produced nothing), and the result is evaluated through the constrained evaluator, with only min/max exposed as extra names, then clamped between 0 and the criteria’s own maximum score.
3. Combine criteria into a period total. The period’s total_score is the sum, over every criteria row, of score times weight divided by 100 — which is why weights are required to sum to exactly 100: a set summing to less would silently under-report.
4. Aggregate across periods into a composite score. The scorecard walks every submitted period, newest first, indexing from 0. For each, its own weighting formula is evaluated twice through the same substitution mechanism — once with the period’s actual total_score, once with a hypothetical maximum of 100 — producing a weighted actual and a weighted ceiling. The composite supplier_score is 100 times the sum of weighted actuals over the sum of weighted ceilings; with no submitted periods, it defaults to a perfect 100.
5. Look up the standing band containing that composite score, and copy its status label, color, notification flags, and all four prevent/warn flags onto the scorecard — and, in the same step, write those same flags directly onto the linked supplier record, so a submission gate never has to consult the scorecard at all.

The dynamic-import mechanism in step 1 is the part of this pipeline most likely to be misread. A dotted path is resolved by importing its first component and walking the rest as attribute lookups — unrestricted, with no allow-list and no sandboxing of what the resolved callable does once invoked. Its only safety net is a save-time check that the import succeeds, which validates importability, not intent. The practical constraint is administrative, not architectural: only a privileged role can create or edit a Variable record, and this resolution re-runs on every save of every period (daily, at minimum, given the scheduled sweep), not once at definition time.

3.3 Error handling

• Criteria weight mismatch: a scorecard, or a period copied from it, refuses to save if its weights do not sum to exactly 100.
• Standing band gaps or overlaps: a scorecard refuses to save if its bands do not exhaustively and exclusively cover 0–100.
• Unresolvable variable reference: a formula placeholder with no matching variable by that name is rejected at save time, naming the missing reference.
• Formula evaluation failure: a malformed criteria or weighting formula blocks the operation with a message naming the offending formula, rather than silently defaulting its score to zero.
• Concurrent period creation: the backfill’s overlap check is a query against already-submitted periods, not a lock — two near-simultaneous triggers rely on that query alone, not a database-level uniqueness constraint.

4. Scale and Reliability

• Recomputation cost is proportional to submitted-period count, not raw transaction volume. The composite-score aggregation walks every submitted period on every save, re-evaluating each one’s weighting formula; a supplier with years of periods pays that cost every time, not once.
• The daily sweep is a full, unprioritized scan across every scorecard in the system, independently checked for missing periods.
• Variable resolution happens fresh on every period validation, not once at creation. Any re-save re-runs every variable’s underlying query (or arbitrary code) — an expensive aggregate is paid for repeatedly, not amortized.
• No distributed lock protects the period-overlap check, so two near-simultaneous triggers rely on query timing rather than a guaranteed-exclusive check.
• Propagation to the supplier record is a direct, synchronous write inside the same save that recomputes the scorecard — no queue or eventual-consistency window between a standing change and the gates that read it.

5. Trade-off Analysis

6. What to Revisit as the System Grows

• Constrain the variable dotted-path mechanism to an explicit allow-list rather than an unrestricted import, once more than a small trusted group can gain write access to a Variable record.
• Replace the overlap query with a database-level uniqueness guarantee for period date ranges, so a duplicate from near-simultaneous triggers becomes structurally impossible, not merely unlikely.
• Bound or cache the per-save variable recomputation, especially for expensive aggregate queries, once scorecards accumulate enough periods that a routine save becomes noticeably slow.
• Give the standing-flag duplication a single source of truth, removing the possibility that the scorecard’s and the supplier’s copies ever disagree.
• Batch or prioritize the daily sweep by how overdue each scorecard’s periods are, once scorecard count makes a full unconditional scan a meaningful cost.