Materialized Index

How the materialized field value index powers fast filters, when it updates automatically, and rebuilding it via POST /v1/search/materialize after bulk loads.

The materialized index pre-computes and stores extracted field values for every document — typed per field, one row per document/field pair — enabling sub-second filter queries even on large workspaces. The document filter (POST /v1/documents/filter), omnisearch, field autocomplete, and field values endpoints all execute against it, which is why their counts and results always agree with each other.

Each index row links a document to a field registry entry and carries the value in a typed slot — text, number, date, or boolean — matching the field's data type. The typing is what makes filter operators behave correctly without client-side casting: between on a date field compares dates, gt on an amount compares numbers, and the field_values map the filter inlines exposes the same typed slots back to you.

For normal operation you never touch it: single-document ingestion materializes values automatically during post-extraction processing. A manual rebuild via POST /v1/search/materialize exists for the cases where reads lag writes — after bulk ingestion of many documents at once, or after schema changes that redefine which fields exist. A stale index shows up as missing documents in filter results, missing suggestions in autocomplete, or a field whose occurrenceCount is high while its documentCount is 0.

The rebuild walks every completed document in the workspace in batches, upserting its values — it is idempotent, safe to re-run, and skips documents whose extraction has not finished (they materialize when they complete). The call is synchronous: the response returns once the backfill has walked the corpus, with processed reporting how many documents were materialized. A document that fails to materialize is logged and skipped rather than aborting the run, so processed can be lower than your corpus size.

curl -X POST https://api.talonic.com/v1/search/materialize \
  -H "Authorization: Bearer $TALONIC_API_KEY"

Response

{
  "status": "complete",
  "message": "Materialization triggered for all documents.",
  "processed": 1842
}

Errors

In a typical bulk-ingestion workflow: upload the batch, wait until the documents report completed, then trigger one materialize call. After it returns, filter, autocomplete, field-values, and omnisearch all reflect the newly ingested data consistently.