Comparisons

List every per-cell N-Shot comparison for a job run: per-shot values with confidence and source text, agreement status and score, overrides, and judgements.

GET /v1/jobs/runs/{runId}/nshot/comparisons retrieves every per-cell comparison for a job run — one entry per document-field pair that the N-Shot evaluation covered. Each comparison shows the value each shot produced, the agreement status (green/yellow/red) with its numeric agreement_score, the majority_value, the comparison_method used, and any override or judge decision already applied. Use it to drill into exactly where extraction diverges across shots.

Each entry in the values array carries more than the raw value: shot_number, value, the shot's own extraction confidence, and source_text — the document passage the shot extracted the value from. When two shots disagree, comparing their source_text usually reveals whether they read different regions of the document (a real ambiguity) or read the same region differently (a formatting or normalization issue you can fix with a field instruction).

The comparison_method is chosen per field from the field's data type: exact for dates and numbers (string equality after normalization), fuzzy for short text (Jaro-Winkler similarity), and semantic for long text (an LLM scores whether the values mean the same thing). A yellow status on a semantic field is softer evidence of divergence than a yellow on an exact field, so triage exact-method disagreements first.

Most integrations start with GET /v1/jobs/runs/{runId}/nshot/summary to check the overall agreement_rate, then fetch this list and filter for status: "red" and status: "yellow" entries. Use the override and judgement fields to track review state: a non-null override means a value was already corrected, and a judgement with accepted: null is a pending LLM recommendation still awaiting a decision via the judge-decision endpoint.

GET/v1/jobs/runs/{runId}/nshot/comparisons

Request

curl https://api.talonic.com/v1/jobs/runs/a1b2c3d4-e5f6-7890-abcd-ef1234567890/nshot/comparisons \
  -H "Authorization: Bearer tlnc_your_api_key"

Response

Response fields

dataarrayArray of comparison objects, ordered by created_at ascending.

data[].idstringComparison UUID.

data[].run_idstringJob run UUID.

data[].document_idstringDocument UUID.

data[].field_namestringField name being compared.

data[].statusstringAgreement status: green (unanimous), yellow (>50% agree), or red (≤50% agree).

data[].agreement_scorenumberFraction of shots agreeing with the majority value (0-1).

data[].majority_valuestring | nullThe value agreed on by the largest group of shots.

data[].comparison_methodstringexact, fuzzy, or semantic — chosen from the field's data type.

data[].valuesarrayPer-shot entries: shot_number, value, confidence, source_text.

data[].overrideobject | nullOverride record if a value was manually selected or a judge decision was accepted.

data[].judgementobject | nullLLM judge recommendation (recommended_shot, recommended_value, reasoning, accepted).

data[].created_atstringISO 8601 creation timestamp.

data[].links.selfstringURL to retrieve this specific comparison.

Response

{
  "data": [
    {
      "id": "5f8e2b1a-3c4d-4e5f-8a9b-0c1d2e3f4a5b",
      "run_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
      "document_id": "d4e5f6a7-b8c9-0123-defa-234567890123",
      "field_name": "invoice_number",
      "status": "green",
      "agreement_score": 1.0,
      "majority_value": "INV-2024-001",
      "comparison_method": "exact",
      "values": [
        { "shot_number": 1, "value": "INV-2024-001", "confidence": 0.98, "source_text": "Invoice No: INV-2024-001" },
        { "shot_number": 2, "value": "INV-2024-001", "confidence": 0.97, "source_text": "Invoice No: INV-2024-001" },
        { "shot_number": 3, "value": "INV-2024-001", "confidence": 0.99, "source_text": "Invoice No: INV-2024-001" }
      ],
      "override": null,
      "judgement": null,
      "created_at": "2026-07-14T10:32:00.000Z",
      "links": {
        "self": "/v1/jobs/runs/a1b2c3d4-e5f6-7890-abcd-ef1234567890/nshot/comparison?document_id=d4e5f6a7-b8c9-0123-defa-234567890123&field_name=invoice_number"
      }
    }
  ]
}

The list is not paginated: it returns every comparison for the run, ordered by creation date ascending. A run with many documents and fields can produce a large response, so filter client-side by status once fetched. Comparisons for documents hidden from you by source visibility rules are excluded.

Errors

Error responses

401unauthorizedMissing or invalid API key.

404not_foundNo job run with this ID exists for your organization.

429rate_limitedDaily request quota for your tier reached. The counter resets at midnight UTC.

Frequently asked questions

What comparison methods are used?+

Three, chosen per field from its data type: exact (string equality after normalization) for dates and numbers, fuzzy (Jaro-Winkler similarity) for short text, and semantic (an LLM scores meaning equivalence) for long text. The method appears on each comparison as comparison_method.

How do I find comparisons that need attention?+

Filter for status: "red" comparisons first (half or fewer shots agree), then status: "yellow" (a majority exists but not unanimous). Green comparisons are unanimous and typically need no review. Within a status, exact-method disagreements are the strongest signal of real instability.

How do I tell which comparisons have already been reviewed?+

Check the override and judgement fields on each comparison. A non-null override means a value was manually selected or a judge recommendation was accepted; a judgement with accepted: null is a pending LLM recommendation still awaiting a decision.

What is source_text on each shot value for?+

It is the document passage the shot extracted the value from. When shots disagree, matching source_text with differing values points at a normalization problem (fixable with a field instruction), while differing source_text means the shots read different parts of the document — a genuine ambiguity in the source.

Comparisons

Response

Errors

Frequently asked questions

Related