Skip to main content

Judge Decision

Submit a judge decision (human or AI) for an N-Shot comparison to record which candidate produced the correct extraction result.

POST/v1/jobs/runs/{runId}/nshot/judge-decision

Response

Response fields

idstringComparison UUID.
run_idstringJob run UUID.
document_idstringDocument UUID.
field_namestringField name.
statusstringAgreement status.
majority_valueanyOriginal majority value.
valuesarrayPer-shot values.
overrideobject | nullOverride record applied if accepted is true.
judgementobjectUpdated judgement record.
judgement.recommended_shotintegerShot number the LLM recommended.
judgement.recommended_valueanyValue the LLM recommended.
judgement.acceptedbooleanWhether the recommendation was accepted.
judgement.decided_bystringAlways "api" when decided via this endpoint.
judgement.decided_atstringISO 8601 timestamp of the decision.

Response

{
  "id": "cmp-uuid-1",
  "run_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "document_id": "doc-uuid-1",
  "field_name": "total_amount",
  "status": "yellow",
  "agreement_score": 0.6667,
  "majority_value": "12450.00",
  "comparison_method": "exact",
  "values": [
    { "shot_number": 1, "value": "12450.00" },
    { "shot_number": 2, "value": "12450.00" },
    { "shot_number": 3, "value": "12,450.00" }
  ],
  "override": {
    "selected_shot": 1,
    "actor_id": "judge",
    "overridden_at": "2024-09-14T11:05:00.000Z",
    "from_value": "12450.00",
    "to_value": "12450.00"
  },
  "judgement": {
    "recommended_shot": 1,
    "recommended_value": "12450.00",
    "accepted": true,
    "decided_by": "api",
    "decided_at": "2024-09-14T11:05:00.000Z"
  },
  "created_at": "2024-09-14T10:32:00.000Z",
  "links": {
    "self": "/v1/jobs/runs/a1b2c3d4-e5f6-7890-abcd-ef1234567890/nshot/comparison?document_id=doc-uuid-1&field_name=total_amount"
  }
}

Errors

Error responses

401unauthorizedMissing or invalid API key.
404not_foundNo job run with this ID exists for your organization, or no comparison found for the given document_id and field_name.
429rate_limitedToo many requests. Retry after the period indicated in the Retry-After header.

Related

Summary
Override