You asked
Fitting, for a rubric literally called “You asked” — this one arrived almost word for word from an onboarding team. A model answer with no source isn’t a check, and it’s dangerous in both directions equally. “Clean,” with no citation, can mean the model simply doesn’t know about a listing added last week. A confident “yes, there’s a match,” with no citation, can just as easily mean the model is recalling a list from years ago — a sanction lifted yesterday can look, to a model with no source, exactly like one still in force. Both answers sound equally sure of themselves. The only thing that tells them apart is a source and a list-state date — not whether the guess happened to be right.
Two directions of the same failure
The model didn’t look at a list — it recalled something that resembles one. A confident, well-written answer that never touched a real source is the most dangerous thing in compliance, because it’s indistinguishable from a real check until someone verifies it. And it fails symmetrically: understating risk (“clean”) lets a real violator through; overstating risk (“there’s a match”) blocks a legitimate counterparty on nothing. Both are real business losses — they just point in opposite directions.
Nothing about a model sounding confident tells you which direction it failed in, or whether it failed at all. That’s exactly why a citation matters more than the answer itself: the citation is the only part of the response that can actually be checked.
Where a confident “clean” goes wrong
Several exchanges were added to the EU’s Russia sanctions regime very recently — real, current designations, live in the platform’s own source data. A model trained even a couple of months earlier answers “nothing found” with complete confidence, simply because the designation hadn’t happened yet when its training data was collected. That isn’t the model making an error in the usual sense — it’s answering from an accurate, outdated picture of the world, without flagging that the picture is outdated.
The gap isn’t a rare edge case. Sanctions lists change constantly — every week brings new designations across multiple regimes — and a model’s knowledge is frozen at whatever point its training stopped.
Where a confident “yes” goes wrong — the same way
One individual and one company sat on the World Bank’s debarment list for over fourteen years — listed in 2011, removed only yesterday. A model that “remembers” that listing is entirely correct about history and entirely wrong about the present — and to whoever reads the answer, that’s indistinguishable from the model simply inventing a match. A confident “yes” with no list-state date is no more trustworthy than a confident “no” without one.
This is why the platform always returns an as-of date alongside every match or clearance, and why re-screening as of a specific historical date is a supported, separate operation — not the same question as “is this true right now.”
Reading any screening answer, in either direction
- A source and a list-state date belong in every answer — not only when the result happens to be “clean”
- “The model said yes, with no source” is exactly as unverifiable as “the model said no, with no source”
- Freshness cuts both ways — a new listing and a lifted one can both fall outside what a model knows
- Ask what the source shows as of what date — not what the model thinks