Validation Stages

Manage a Spec validation stages: positional quality checkpoints that validate the pipeline phases before them. Create, reorder, and replay blocking stages.

These endpoints configure a Spec (the same entity as /v1/schemas). A Spec is a user schema plus the rail of stages that govern how documents flow through the One Engine pipeline: source, field registry, extraction, resolution, validation, and data product. You configure a Spec with the endpoints here, then you run it by compiling the rail into a pipeline via POST /v1/pipelines. Field-level normalization and transform config is not set on fields here: it lives in Data Policies (/v1/data-policies), which the resolution stage references.

A validation stage is a positional quality checkpoint. It validates the output of the pipeline phases that ran before it at its rail position, so where a stage sits in the order determines what it checks. Several checkpoints may appear in one rail, each guarding a different segment. Stages are returned ordered by position and are appended to the end of the sequence on create.

Each stage carries a severity: info logs only, warning flags failing cells, and blocking halts the failing field and its dependents. A blocking stage also carries a blockingConfig that decides the pass threshold, the critical fields, the failure scope, and the on_block routing (review_queue, auto_retry, or hold). The fieldScope narrows a stage to a subset of fieldKeys, or leaves it validating every field.

Response

A stage object reflects the persisted validation_stages row. ordinal is the stage position in the rail. blocking_config is null unless severity is blocking.

[
  {
    "id": "f1e2d3c4-b5a6-7890-abcd-ef1234567890",
    "user_schema_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    "name": "Totals must reconcile",
    "description": "Block delivery when line items do not sum to the invoice total.",
    "severity": "blocking",
    "config": {
      "businessRules": { "enabled": true }
    },
    "blocking_config": {
      "threshold": { "type": "percentage", "min_pass_rate": 0.95 },
      "critical_fields": ["total", "line_items"],
      "scope": "delivery_only",
      "on_block": "review_queue"
    },
    "field_scope": "all",
    "field_keys": [],
    "ordinal": 0,
    "created_at": "2024-09-14T10:32:00.000Z",
    "updated_at": "2024-09-14T10:32:00.000Z"
  }
]

Reorder returns { "reordered": true }. Delete returns { "deleted": true, "id": "<stageId>" }. Replay-blocking returns a summary of the affected fields (and, when dryRun is true, the counts that would have changed without writing).

Errors