Wanted Sources & Discovery

See which datasets your unanswered needs name as missing, and what the discovery job tried for them: site lookups, price sheets, and why a lookup failed.

Some needs name the dataset the library lacks in subject.wanted. GET /v1/data-library/needs/wanted counts those per dataset and category with the number of contracts they concern, most needed first. It is the list of what to load or scrape next: the datasets whose absence blocks the most work. scope restricts it to one scope. Only unanswered needs count, so a dataset leaves the list once the library holds it and the apps report the need no more.

Each entry carries a discovery object that reports what the discovery job last made of the source, or null when the job has not tried it. For an ordinary dataset it holds the job's state: status, problem, detail, attempts, next_attempt_at and updated_at. For the grid-operator register (de.grid-operators) it summarises the site lookups instead, as { sites, found, failed }, where failed counts the failed lookups per problem. The two shapes differ because the register is built from one lookup per site, while a price sheet is one source.

{
  "data": [
    {
      "dataset": "grid.power.price-sheet:example-netz:2026",
      "category": "grid_fee",
      "needs": 12,
      "contracts": 5,
      "discovery": {
        "status": "failed",
        "problem": "no_sheet",
        "detail": "No page on the operator site links a sheet for 2026",
        "attempts": 2,
        "next_attempt_at": "2026-10-27T03:47:00.000Z",
        "updated_at": "2026-10-20T03:47:12.000Z"
      }
    }
  ]
}

What the Discovery Job Tried

GET /v1/data-library/discovery lists the discovery job's attempts for the workspace's wanted sources, ordered by last update, latest first. Two kinds of target appear: the workspace's own site lookups for the grid-operator register (target is de.grid-operators|<energy>|<site> and shared is false) and the shared price-sheet lookups (target is the wanted dataset key and shared is true). status is pending, found or failed. attempts and next_attempt_at show the backoff: a failed lookup is tried again later, not at once.

When a lookup fails, problem says why and detail adds the specifics. Typical values are no_postcode, no_location, no_operator, gas_operator_unknown, no_site, no_sheet, needs_ocr, provisional_only and error; the set is descriptive and may grow, so treat an unknown value as a failure with the text in detail. result holds what was found: the sheet URL, the page that links it, the dataset created, the operator. The job never invents a source: it picks a URL only after fetching it and checking that its text matches what was wanted.

curl "https://api.talonic.com/v1/data-library/discovery?status=failed&limit=50" \
  -H "Authorization: Bearer $TALONIC_API_KEY"