Create Validation Run

Start a validation run with POST /v1/validation/runs, registering a benchmark of one job run against a golden sample for per-field extraction accuracy.

POST /v1/validation/runs registers a validation run: a benchmark that compares the output of one job run (Structuring Run) against a golden sample of verified expected values. When the run executes, the engine classifies every document-field pair as exact, partial, or mismatch (plus missing-value categories), and an optional LLM judge pass adds semantic verdicts. This measures accuracy after extraction, unlike the in-pipeline checks that gate results before delivery.

The endpoint validates both references before inserting: dataspace_run_id is checked first (404 "Job run ... not found."), then golden_sample_id (404 "Golden sample ... not found."); both must be UUID v4 or the request fails with 400 VALIDATION_ERROR. The job run must be completed (a 400 validation_error otherwise) and must use the same schema as the golden sample. On success the comparison runs in the request and the run is returned already completed, with accuracy, total_comparisons and field_accuracy populated; if the engine hits an error the run is returned as failed with error_message.

This is the same engine the platform's Benchmarks surface (Review → Benchmarks) runs, so scores from the API and the dashboard agree. A golden sample with a few hundred values compares in well under a second; the request returns when the comparison is done, so there is nothing to poll.

For a meaningful benchmark, validate a completed Structuring Run whose schema matches the golden sample's user_schema_id — the engine compares field names defined by that schema, so a schema mismatch produces mostly missing comparisons rather than an error at creation time. If name is omitted, the run is named Validation YYYY-MM-DD from the creation date; use explicit names when creating several runs a day, since the list endpoint has no filters.

{
  "dataspace_run_id": "b2c3d4e5-f6a7-8901-bcde-f12345678901",
  "golden_sample_id": "c3d4e5f6-a7b8-9012-cdef-123456789012",
  "name": "Q1 Invoice accuracy check"
}

curl -X POST https://api.talonic.com/v1/validation/runs \
  -H "Authorization: Bearer tlnc_..." \
  -H "Content-Type: application/json" \
  -d '{
    "dataspace_run_id": "b2c3d4e5-f6a7-8901-bcde-f12345678901",
    "golden_sample_id": "c3d4e5f6-a7b8-9012-cdef-123456789012",
    "name": "Q1 Invoice accuracy check"
  }'

Response

{
  "id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "name": "Q1 Invoice accuracy check",
  "status": "completed",
  "dataspace_run_id": "b2c3d4e5-f6a7-8901-bcde-f12345678901",
  "golden_sample_id": "c3d4e5f6-a7b8-9012-cdef-123456789012",
  "accuracy": 0.9412,
  "total_comparisons": 68,
  "matched_documents": 17,
  "unmatched_golden": 0,
  "unmatched_dataset": 2,
  "field_accuracy": { "invoice_number": { "total": 17, "exact": 17, "partial": 0, "mismatch": 0, "missing_actual": 0, "missing_expected": 0, "both_empty": 0, "accuracy": 1 } },
  "error_message": null,
  "created_at": "2024-09-14T10:32:00.000Z",
  "completed_at": "2024-09-14T10:32:01.000Z",
  "links": {
    "self": "/v1/validation/runs/a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    "results": "/v1/validation/runs/a1b2c3d4-e5f6-7890-abcd-ef1234567890/results"
  }
}

Errors