Skip to main content

Document Evaluations

Evaluate documents against the rules of the ON frameworks, poll the status, and read per-rule verdicts with reasons and quotes via /v1/compliance/evaluations.

An evaluation judges one document against the rules of the ON frameworks. Each rule receives a verdict, pass, fail, not_applicable or unclear, with a reason and a quote from the document. The documents an app evaluates form the Evaluations list: documents tagged compliance-evaluation, whether picked from the workspace, uploaded in the Compliance app, or evaluated through these routes. Evaluating a document puts it on the list.

An evaluation row carries counts (rule_count, pass_count, fail_count, not_applicable_count, unclear_count) and a status of processing, waiting, completed or error. waiting means the evaluation is offered to the workspace's external agent. outdated is true when the ON frameworks, their rules or the rule count differ from the ones the evaluation ran with, which is the signal to evaluate again. A document stored without text has nothing to judge; its evaluation ends with error at once.

A document that is itself a framework is refused (422), and unknown document ids return 404. With no framework ON, evaluating is refused with 422. A document whose evaluation is still running is not evaluated twice: the running evaluation is returned. Duplicate documents are read through the document they duplicate.

POST/v1/compliance/evaluations

Body fields

document_ids*string[] (uuid)1 to 500 documents. The configured limit (500 by default) applies on top.

cURL: Evaluate two documents

curl -X POST "https://api.talonic.com/v1/compliance/evaluations" \
  -H "Authorization: Bearer $TALONIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"document_ids": ["5b7e2a91-6c44-4d0f-9a3e-71c8d2f0b6a5", "a83c06d2-9e1f-4b57-8c20-4d7f1e95a6b3"]}'

Response: 201 Evaluations started

{
  "data": [
    {
      "id": "c2f1a7d0-84b3-47e6-9d15-0be3a6c97f21",
      "document_id": "5b7e2a91-6c44-4d0f-9a3e-71c8d2f0b6a5",
      "set_id": null,
      "status": "processing",
      "decided_by": null,
      "error": null,
      "framework_ids": ["9d0c4f6e-3b1a-4c53-8f0e-2a7b6c1d5e40"],
      "rule_count": 28,
      "pass_count": 0,
      "fail_count": 0,
      "not_applicable_count": 0,
      "unclear_count": 0,
      "outdated": false,
      "member_ids": null,
      "created_at": "2026-10-20T09:12:30.000Z",
      "finished_at": null
    }
  ]
}
POST/v1/compliance/evaluations/documents

Body fields

document_ids*string[] (uuid)1 to 500 documents.

The difference from POST /v1/compliance/evaluations is the empty-framework case. Adding documents to the list works before any framework is ON, so documents can be queued first; the plain evaluate route is refused with 422 until a framework is ON. added reports how many documents were put on the list, and evaluations holds the evaluations started.

POST/v1/compliance/evaluations/all

Body fields (all optional)

dry_runbooleanOnly report how many documents would be evaluated.
limitintegerAt most this many documents start per call, 1 to 500; the rest follow on the next call.

Response fields

dry_runbooleanEchoes the request.
candidatesintegerDocuments that need an evaluation.
startedintegerEvaluations started; 0 on a dry run.
document_idsstring[] (uuid)The documents selected after limit.

cURL: Dry-run a catch-up evaluation

curl -X POST "https://api.talonic.com/v1/compliance/evaluations/all" \
  -H "Authorization: Bearer $TALONIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"dry_run": true, "limit": 100}'
GET/v1/compliance/evaluations
GET/v1/compliance/evaluations/status

Response fields

processing / waiting / completed / errorintegerEvaluations by status.
outdatedintegerCompleted evaluations that are outdated.
failing_documentsintegerCompleted evaluations with at least one failed check.
failing_rulesintegerFailed checks over those evaluations.
groupsobjectThe same counts over document sets: processing, waiting, completed, error, outdated, failing and failing_rules.
GET/v1/compliance/evaluations/documents/{documentId}

Result fields

results[].verdictstringpass, fail, not_applicable or unclear.
results[].reasonstring | nullWhy the rule got this verdict.
results[].quotestring | nullThe passage the verdict rests on.
results[].rule_idstring | nullThe rule, or null for a consistency finding of a set evaluation.
results[].rule_code / rule_text / obligationstringCopied from the rule as judged, so they stay after the rule is edited or deleted.
results[].framework_document_id / framework_namestringThe framework the rule belongs to.
results[].needs_checkbooleanTwo AI readings disagreed and a third decided; a person should check this result.

Response: 200 Document evaluation (excerpt)

{
  "evaluation": {
    "id": "c2f1a7d0-84b3-47e6-9d15-0be3a6c97f21",
    "document_id": "5b7e2a91-6c44-4d0f-9a3e-71c8d2f0b6a5",
    "set_id": null,
    "status": "completed",
    "decided_by": null,
    "error": null,
    "framework_ids": ["9d0c4f6e-3b1a-4c53-8f0e-2a7b6c1d5e40"],
    "rule_count": 28,
    "pass_count": 19,
    "fail_count": 2,
    "not_applicable_count": 6,
    "unclear_count": 1,
    "outdated": false,
    "member_ids": null,
    "created_at": "2026-10-20T09:12:30.000Z",
    "finished_at": "2026-10-20T09:14:52.000Z",
    "results": [
      {
        "id": "3e8d5c71-0a42-4b96-b1f8-c7d92e064a35",
        "rule_id": "0f3d8a52-1b6c-4e97-a2d4-8c5e91b07f36",
        "framework_document_id": "9d0c4f6e-3b1a-4c53-8f0e-2a7b6c1d5e40",
        "framework_name": "Supplier Code of Conduct",
        "rule_code": "R-07",
        "rule_text": "The supplier must report security incidents within 72 hours.",
        "obligation": "must",
        "verdict": "fail",
        "reason": "The contract allows 14 days for incident reports.",
        "quote": "Incidents are reported within 14 days of discovery.",
        "document_id": null,
        "needs_check": false
      }
    ]
  }
}
DELETE/v1/compliance/evaluations/documents/{documentId}
An unclear verdict is not a pass. It means the document does not state enough to decide, for instance because the rule needs another document. Judge such rules on a document set, where the other documents are part of the evaluation.

Frequently asked questions

Why is my evaluation marked outdated?+
The ON frameworks, one of their rules or the rule count changed after the evaluation ran, so its verdicts no longer describe the current rules. POST /v1/compliance/evaluations/all evaluates every listed document that is outdated, never evaluated or ended in an error.
What does needs_check mean?+
When two AI readings of a rule disagree, a third decides and the result is marked needs_check: true, so a person knows to look at it first.
Why is evaluating refused with 422?+
Either no framework with rules is ON, or one of the documents is itself a framework; frameworks are not evaluated. Unknown document ids return 404.
What happens to an evaluation of a document stored without text?+
A store-only upload has no text to judge, so its evaluation ends with status error at once. Duplicate documents are read through the document they duplicate.

Related

Compliance Overview
Document Sets
Issues and Export
Compliance Frameworks