Skip to main content

Create Validation Run

Start a validation run with POST /v1/validation/runs, comparing a job run against a Ground Truth dataset for per-field extraction accuracy scores.

POST /v1/validation/runs starts a validation run: a benchmark that compares the output of one job run against a Ground Truth dataset of verified expected values. The benchmark engine compares each extracted value to the expected value, computing exact match, fuzzy match, and similarity scores; an LLM judge provides a semantic verdict for ambiguous cases. This measures accuracy after extraction, unlike the in-pipeline checks that gate results before delivery.

Validation runs start in pending status and move to running as comparisons are performed. Once complete, the accuracy field contains the overall score and per-field results are available via the Results endpoint.

Both golden_sample_id and dataspace_run_id must belong to your organization. The API returns 404 if either resource is not found.
POST/v1/validation/runs

Body parameters

golden_sample_id*uuidGround Truth dataset to validate against.
dataspace_run_id*uuidJob run to validate.
namestringOptional human-readable name. Defaults to "Validation YYYY-MM-DD".

Request body

{
  "dataspace_run_id": "b2c3d4e5-f6a7-8901-bcde-f12345678901",
  "golden_sample_id": "c3d4e5f6-a7b8-9012-cdef-123456789012",
  "name": "Q1 Invoice accuracy check"
}

Response

Response fields (201 Created)

idstringValidation run UUID.
namestringRun name.
statusstringInitial status, always pending.
dataspace_run_idstring | nullUUID of the job run being validated.
golden_sample_idstringUUID of the Ground Truth dataset.
accuracynumber | nullAlways null until the run completes.
total_comparisonsinteger | nullAlways null until the run completes.
created_atstringISO 8601 creation timestamp.
completed_atstring | nullAlways null until the run completes.
linksobjectRelated resource URLs (self, results).

Response (201 Created)

{
  "id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "name": "Q1 Invoice accuracy check",
  "status": "pending",
  "dataspace_run_id": "b2c3d4e5-f6a7-8901-bcde-f12345678901",
  "golden_sample_id": "c3d4e5f6-a7b8-9012-cdef-123456789012",
  "accuracy": null,
  "total_comparisons": null,
  "created_at": "2024-09-14T10:32:00.000Z",
  "completed_at": null,
  "links": {
    "self": "/v1/validation/runs/a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    "results": "/v1/validation/runs/a1b2c3d4-e5f6-7890-abcd-ef1234567890/results"
  }
}

Errors

Error responses

401unauthorizedMissing or invalid API key.
404not_foundJob run or Ground Truth dataset not found, or they do not belong to your organization.
429rate_limitedToo many requests. Retry after the period indicated in the Retry-After header.

Related

List Validation Runs
Validation Results
Get Job Results