We audited 1,000 real Scoopkit events and found 2 real bugs in our own classification and clustering code. One could publish placeholder text as a real headline. The other could silently cause 2 different events to merge into one. Both are fixed and verified live as of this week.
Verified 2026-08-30. Sample: 1,000 events pulled from the public API. Method: automated checks across every field, plus a manual code review of every place the pipeline trusts a model's JSON response.
Facts
| Bug | What it did | How often | Status |
|---|---|---|---|
| Placeholder echo | Overwrote a real headline/summary/orgs/models with literal "..." | 1 of 1,000 events | Fixed |
| Boolean parsing | Could read "false" as true in merge decisions | Found by code audit, not by sampling | Fixed |
| Date field drift | A details field showed the wrong year | 2 of 1,000 events | Not a bug, left as-is |
The placeholder-echo bug
Scoopkit's classifier prompt shows the model an example JSON shape, using "..." as placeholder syntax for string fields, the same convention you'd see in API documentation. One event's classification call returned that placeholder literally instead of real content, and our code had no check for it. The result: an event about a real Google speech-to-text model release ended up with "..." as its headline, summary, orgs, and models, live on the public API.
We caught this by sampling 1,000 events and checking every field for exact placeholder matches. Only 1 event out of 1,000 showed it, but it was completely broken, not just cosmetically off.
The fix treats "..." and a handful of similar placeholder strings as invalid input for those 4 fields, falling back to the event's existing content instead of overwriting it with garbage. We re-ran classification on the one broken event afterward; it now shows the real headline and a correctly assigned category.
The boolean parsing bug
This one didn't show up in the 1,000-event sample. We found it by reading the code that decides whether 2 articles describe the same real-world event.
That decision comes down to one field in a model's response: same_event: true or same_event: false. Our code checked it with bool(data.get("same_event")). In Python, bool("false") is True, because it's a non-empty string. If a model answered with the string "false" instead of a real JSON boolean, which weaker models do sometimes, our code would read it as True and could merge 2 different events together.
We didn't find a live example of this actually happening, but the code path was real and reachable in both the clustering tie-break and the nightly merge pass. The fix only counts a real boolean true, or the string "true", as true; anything else, including the string "false", now correctly reads as false.
How we verified both fixes
For each bug, we wrote out the exact failure case, the actual bad input that caused it, and confirmed the fixed code handles it correctly. We also ran a normal, valid response through both to confirm nothing broke for the common case. Get a free API key and query /v1/events if you want to see the current classification output yourself.
FAQ
Does this affect data I already pulled from the API?
Only 1 event was affected by the placeholder bug, and it's been corrected. The boolean bug was never confirmed to have caused a real merge, since we found it by code review, not by finding a broken live event.
How did you find these?
The placeholder bug came from a routine data-quality sweep after an unrelated question about a date field. The boolean bug came from auditing every place in the codebase with the same shape of problem, trusting a model's output without checking it first.
Could something like this happen again?
Possibly, in a different form. This is why we sample real data and read the code that handles model output, instead of assuming a validated JSON response means a correct one.