Validation Checkpoints

A validation checkpoint is a gate you place at a position in the rail. It validates the cumulative output of whatever stages precede it, so a checkpoint after Extraction judges raw extraction while a checkpoint after Resolution judges normalized values. You can place several checkpoints in one rail, each scoped to the work before it. A checkpoint has no position of its own: where it runs is decided entirely by where you drop it in the rail.

Each checkpoint combines four core validation mechanisms over the fields in scope. Evidence applies deterministic rules that check a value against the source text. Business Rules evaluates declarative field-level and cross-field predicates deterministically, sending only rules marked semantic to a single batched model call. N-shot re-runs extraction several times and compares the results to measure reproducibility. LLM Judge asks a model to assess correctness. The verdicts are stored per field, and they drive both the review queue and the data product holdback.

A checkpoint can be a warning (record the verdict, ship the value) or blocking (hold failing fields for review). A blocking gate writes a pending-approval cell for each failing field, flips the field state, and routes it to the central review queue or holds it on the pipeline page, according to the gate config. Dependent fields wait until the blocked field is resolved. The verdicts are stored, so a gate flipped from warning to blocking after the fact can be replayed over existing results without re-validating anything.

N-shot is a reproducibility check, not a second opinion. It re-runs the real extraction on the same document N times and compares each run to the value first committed. A field that comes back identical across runs is stable; a field that varies is flagged. When the checkpoint covers resolution, n-shot compares the resolved values, so a difference that normalization erases (1.000,50 versus 1000.5) reads as agreement.

Checkpoints are built from reusable validation stages (gates) configured through the validation-stages endpoints on the schema. A gate carries its severity, field scope, and blocking behavior; the rail decides where it runs. The same gate can appear at more than one checkpoint. An optional "covers phases" control scopes a checkpoint to fields produced by specific preceding phases, so a late checkpoint can validate only the resolution output rather than re-judging everything.

Create a validation stage on a schema

curl -X POST https://api.talonic.com/v1/schemas/3f6c1b7e-2a4d-4c1e-9b8a-5d2e7f9a1c04/validation-stages \
  -H "Authorization: Bearer $TALONIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Required fields present",
    "severity": "blocking",
    "fieldScope": "subset",
    "fieldKeys": ["delivery_date", "total_amount"]
  }'

Reference the stage from a valid rail stage by its ID, and it becomes a checkpoint at that position. Because verdicts are stored, you can tune a gate over time. Run it as a warning first to see what it would catch, then flip it to blocking and replay it over the documents you already processed to push the failures into review, with no re-run and no model calls.

Validation mechanisms fail closed. A validator that cannot run (a model outage, missing reference data, an n-shot re-extraction that returns nothing) emits fail verdicts rather than passing silently. A blocking gate will hold those fields for review, so an infrastructure problem never lets an unchecked value through.

Severity, scope, and additional checks

Every gate carries one of three severities. info is the default and only records verdicts; warning flags failing cells but lets the values ship; blocking holds failing fields and their dependents. A gate validates every schema field (fieldScope: "all", the default) or only the keys you list (fieldScope: "subset" with fieldKeys). Scoping a gate to a handful of fields keeps model-backed mechanisms cheap and stops a noisy optional field from tipping a blocking threshold. When and how a blocking gate actually holds fields, including thresholds, critical fields, and auto-release, is covered in Review Gates.

Beyond the four core mechanisms, a gate can enable further checks in its config. Ground Truth looks up each record in a reference table by a composite match key and compares mapped fields, with optional fuzzy matching and per-field severity overrides. Capture cross-check corroborates low-confidence verbatim values against the document's independent capture, with no model call. Cross-source agreement compares extracted values with the values the Field Registry stage seeded, and needs a Field Registry stage on the rail. Write-time validation rejects a Ground Truth check that is enabled without a reference table, a match key, and at least one mapping, because an empty configuration would never produce a verdict.

To place a gate, add a valid stage to the rail with PUT /v1/schemas/{id}/rail, listing the gate IDs in json.stage_ids. The optional json.target_phases takes transfer, extraction, resolution, or assembly and scopes the checkpoint to cells produced by those phases. The rail write replaces the whole rail, so read it first with GET /v1/schemas/{id}/rail and send it back with your change. An API key cannot change the extraction stage's model or input-mode settings: a rail write that alters them returns a 403, while an unchanged round trip is accepted. Stages sent without an id get one assigned server-side, so a rail composed through the API stays editable in the Spec editor.

Place a checkpoint after Resolution in the rail

curl -X PUT https://api.talonic.com/v1/schemas/3f6c1b7e-2a4d-4c1e-9b8a-5d2e7f9a1c04/rail \
  -H "Authorization: Bearer $TALONIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "rail": [
      { "type": "source", "name": "Source" },
      { "type": "schema", "name": "Extraction" },
      { "type": "resolve", "name": "Resolution", "json": { "policy_ids": ["9c2e4a1b-7d3f-4e6a-8b5c-2f1a0d9e8c7b"] } },
      { "type": "valid", "name": "Totals check",
        "json": { "stage_ids": ["8b1d4e2a-6c3f-4a7b-9e5d-1f2a3b4c5d6e"], "target_phases": ["resolution"] } },
      { "type": "deliver", "name": "Data product" }
    ]
  }'

Replaying a gate works through POST /v1/schemas/{id}/validation-stages/{stageId}/replay-blocking. The stage must already be saved as blocking, otherwise the call returns a 400. Send {"dryRun": true} first: the report counts the documents scanned and affected, the fields that would block, and a per-pipeline breakdown, and it lists why fields were skipped. A field is skipped when it is already in review, when a reviewer already approved or overrode it at that gate, when it has no field state, or when review triage demoted it. Applying the replay writes the held cells, cascades dependents, reopens a complete document as partial, and fires the gate's block notification per document.

replay-blocking dry-run report

{
  "stageId": "8b1d4e2a-6c3f-4a7b-9e5d-1f2a3b4c5d6e",
  "dryRun": true,
  "documentsScanned": 412,
  "documentsAffected": 37,
  "fieldsBlocked": 41,
  "skipped": { "alreadyInReview": 6, "clearedByReview": 3, "noFieldState": 0, "capacityDemoted": 2 },
  "byPipeline": [
    { "pipelineId": "5e8d2c1a-9f7b-4a3e-b6d4-0c1f2e3a4b5c", "pipelineName": "Delivery notes", "documentsAffected": 37, "fieldsBlocked": 41 }
  ]
}

Frequently asked questions

Where in the pipeline does a validation checkpoint run?+

Wherever you place it in the rail. A checkpoint validates the cumulative cell values produced by the stages before it, so a checkpoint after Extraction judges raw extraction and one after Resolution judges normalized values. You can place several checkpoints in a single rail, each scoped to the work before it.

What checks does a checkpoint run?+

Four core mechanisms: Evidence (deterministic rules against source text), Business Rules (declarative field and cross-field predicates, evaluated deterministically), N-shot (re-runs extraction to measure reproducibility), and LLM Judge (model assessment of correctness). Verdicts are stored per field and drive the review queue and data-product holdback.

What does a blocking gate do to a failing field?+

It writes a pending-approval cell copying the latest value, flips the field state to blocked, and routes it to the central review queue or holds it on the pipeline page per the gate config. Dependent fields wait. The value never ships until a reviewer approves, corrects, or overrides it.

Can I make a gate blocking after a run already happened?+

Yes. Because verdicts are stored, flipping a gate from warning to blocking lets you replay it over existing results: the stored verdicts are pushed through the same block-selection the live path uses, with no re-validation and no model calls. A dry-run previews the blast radius before you apply it.

Can I change gates through the API on every Spec?+

Reads work on every Spec. Writes (create, update, delete, reorder, and replay) return 409 spec2_owned when the Spec is authored as a Spec document in the Spec editor, because the next publish would overwrite an out-of-band change. Edit those gates in the Spec editor and publish instead.

Does a warning gate cost the same as a blocking one?+

Yes. Severity only changes what happens to the verdicts, not how they are produced, so a warning gate runs the same mechanisms and makes the same model calls. To reduce cost, narrow the gate with fieldScope subset or scope it with target_phases rather than downgrading its severity.

Validation Checkpoints

Severity, scope, and additional checks

Frequently asked questions

Related