Every Scoopkit event carries a details object holding facts specific to its category: a funding amount, a parameter count, an arXiv ID. For most of this product's life that object had no schema. The classifier prompt asked for "details": {} and said nothing else, so the model invented field names per event. Across 2,716 events that produced 1,957 distinct keys, 1,443 of them used exactly once. A funding amount arrived under 55 different names. details is a paid field, which meant the thing Pro subscribers were buying was the thing nobody could write a parser against. This is what that looked like, what we changed, and what the dry run caught before it reached production.
Figures below were counted directly against the production database on 2026-09-08, before and after the migration, not estimated.
What no schema actually produces
The prompt said this, and nothing more:
`
{
"primary_category": "...",
"subcategory": "CATEGORY.SUBCATEGORY",
"details": {}
}
`
An LLM handed an empty object and no field list will fill it with something reasonable every time. The problem is that "reasonable" is decided fresh on each call. One funding event returns amount_usd. The next returns funding_amount. The next round_size_usd, or raise_amount, or deal_value, or investment_amount_usd.
Every one of those is a defensible name. Collectively they are unusable.
| Before | After | |
|---|---|---|
Distinct details keys | 1,957 | 58 canonical |
| Keys used exactly once | 1,443 | preserved, quarantined |
| Names for a funding amount | 55 | 1 (amount_usd) |
| Money-ish key names on the funding tracker | 83 | 3 |
The failure is quiet, which is why it survived so long. Nothing errors. Every individual event looks fine. It only shows up when you try to compute across events, which is exactly what an API is for and exactly what no single event will ever reveal.
Why a better prompt is not the fix
The obvious response is to list the allowed fields in the prompt. We did that, and it helps. It is not sufficient, because a prompt is a request and not a guarantee. A model that returns round_size instead of amount_usd on one call in fifty has not malfunctioned, and there is no error to catch.
So the contract is enforced twice. The prompt tells the model which fields its chosen category may emit. Then a normalizer runs on write, before anything is stored:
- Aliases collapse onto the canonical name. All 55 amount spellings become
amount_usd. - Numbers get coerced.
"$3.5 billion"becomes3500000000. - Anything unrecognized moves to
details.extrarather than being dropped.
That last rule matters more than it looks. The long tail contained real signal: geopolitical_context, stealth_exit, bankruptcy_type. A strict contract that discarded them would have traded one kind of data loss for another. Read the top level for a stable schema, read extra too if you want everything the model saw.
Only 35.5% of historical key instances promoted to canonical names, and that number is honest rather than flattering. The old prompt never asked for these fields, so the model never emitted many of them. The rest sits in extra, intact.
The three bugs the dry run caught
The migration had to rewrite details on 1,919 existing events. Before running it against production we ran it read-only against a copy and diffed the result against a full backup. That found three defects.
The migration would have failed on its first query. events.id is a text primary key like ev_b0a7e82781b0410b, and the keyset pagination initialized its cursor to the integer 0. Postgres rejects text > integer outright. This one was loud, and it would have cost nothing but time.
Number coercion silently dropped qualifiers. "$3.2B+" parsed to 3200000000, losing the "at least". "~$1.3B" lost its approximation. "$2.66B (2023)" lost the year the figure came from. The parsed number is what makes the field usable, so that stays, but the original string is now kept beside it in extra.
An alias could outrank the canonical key it maps to. In this category both focus and sector normalize to sector. One real event carried both:
`json
{"focus": "Medicare Advantage care coordination", "sector": "healthcare AI"}
`
Dictionary ordering decided the winner, so focus claimed the sector slot and the actual sector value got demoted to extra. Exact canonical names are now processed first. A key already using the contract name always wins its own slot.
Two of those three were silent. The migration would have reported success while quietly degrading data, which is the failure mode that costs the most to discover later.
What changed for callers
If you query FUNDING_AND_DEALS events with a Pro key, amounts are now at details.amount_usd as a number, on every event that has one. Valuations at details.valuation_usd. Investors at details.lead_investors as an array, whether the model wrote lead_investor, investors, or investor. You can get a key and check.
The archive was backfilled rather than left split, so the full 365-day Pro window uses one schema instead of changing shape at the deploy date. The backfill renamed keys only. It did not re-run the classifier, so no published headline, summary, or category moved.
One thing this did not fix: coverage. 43% of funding events still carry no amount at all, because the underlying coverage never stated one. A contract makes present data parseable. It cannot invent absent data, which is why the funding tracker still refuses to publish a dollar total.
FAQ
Is this a breaking change?
Yes, for anyone parsing details field names. It was made deliberately at the point where it cost the least. Free-tier callers never saw details at all, so nothing changed for them.
Why flat keys instead of nesting by category?
An earlier schema draft specified details.funding and details.model sub-objects. It was never implemented, and on review it earned nothing: primary_category already tells you which category produced the fields, so the nesting added a level to every read to restate something the caller already had.
What stops this happening again?
The normalizer runs on write, so a model that ignores the prompt cannot put an unrecognized name into the top level. New field names land in extra, where they are visible and harmless rather than silently fragmenting the schema.
How would I have noticed this myself?
Count distinct keys across events instead of reading individual responses. Per-event the output looks correct, which is what made this survive as long as it did. The aggregate is where it falls apart.