Comparisons
List per-document field comparisons for N-Shot evaluation across job runs. Returns all comparisons for a specific job run.
GET
/v1/jobs/runs/{runId}/nshot/comparisonsResponse
Response fields
dataarrayArray of comparison objects.
data[].idstringComparison UUID.
data[].run_idstringJob run UUID.
data[].document_idstringDocument UUID.
data[].field_namestringField name being compared.
data[].statusstringAgreement status: green, yellow, or red.
data[].agreement_scorenumberNumeric agreement score between 0 and 1.
data[].majority_valueanyThe value agreed upon by the majority of shots.
data[].comparison_methodstringMethod used to compare values (e.g. exact, semantic).
data[].valuesarrayPer-shot values with shot_number and value.
data[].overrideobject | nullOverride record if a value was manually selected.
data[].judgementobject | nullLLM judge recommendation with accepted flag.
data[].created_atstringISO 8601 creation timestamp.
data[].links.selfstringURL to retrieve this specific comparison.
Response
{
"data": [
{
"id": "cmp-uuid-1",
"run_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"document_id": "doc-uuid-1",
"field_name": "invoice_number",
"status": "green",
"agreement_score": 1.0,
"majority_value": "INV-2024-001",
"comparison_method": "exact",
"values": [
{ "shot_number": 1, "value": "INV-2024-001" },
{ "shot_number": 2, "value": "INV-2024-001" },
{ "shot_number": 3, "value": "INV-2024-001" }
],
"override": null,
"judgement": null,
"created_at": "2024-09-14T10:32:00.000Z",
"links": {
"self": "/v1/jobs/runs/a1b2c3d4-e5f6-7890-abcd-ef1234567890/nshot/comparison?document_id=doc-uuid-1&field_name=invoice_number"
}
}
]
}Errors
Error responses
401unauthorizedMissing or invalid API key.
404not_foundNo job run with this ID exists for your organization.
429rate_limitedToo many requests. Retry after the period indicated in the Retry-After header.