The pick that broke the spell was a roughly $1.27 option that, on paper, looked perfectly tradeable. Tight spread, clean fill, in it goes. Except the spread was not tight. The spread did not exist. There was no live quote on that contract at all, and somewhere three layers down my own code had decided that the absence of a quote was close enough to a zero-width spread, so it wrote one in.
I had built a filter to throw out illiquid contracts with wide spreads. For months that filter had been doing the opposite of its job on a large slice of the universe, because the worst, least-quoted contracts were arriving with the most attractive spread number possible: 0.0 percent. The filter waved them through first.
Here is the actual mechanism. The options data client tried to read a bid and an ask. When the API returned nothing (common on thinly traded contracts, which is exactly the population you most want to filter), the client fell back to the day's low and high. On a contract that had traded once that day, low equals high, so the synthesized spread came out to zero. A missing quote became a perfect quote. The filter that was supposed to protect me was being handed the cleanest possible input on precisely the contracts it existed to reject.
I want to be honest about what I got wrong, because it is the whole point. I had been treating my own pipeline's field values as ground truth. I scrutinized every word the language model produced. I had eval harnesses, leakage checks, and a paranoid review step for anything the model said. And the model was downstream of a number my own code had fabricated. I was auditing the witness and ignoring the evidence locker.
So I ran the codebase the way I had been running the model: adversarially. Instead of asking "does this work," I asked an agent to assume every field was wrong and prove it, against both the code and the stored data, not against my mental model of the code. That single reframe is what turned up the rest.
There were thirteen. The spread synthesis was the loudest, but the one that probably cost the most was quieter: a scoring step that flipped a smart-money divergence signal was running after the conviction sub-scores instead of before. The ordering meant the flip almost never propagated. In practice it was suppressing somewhere around 87 percent of the setups I most wanted to surface. Nothing crashed. The score just came out low, every day, for the best candidates, and I had no reason to suspect a number that looked like a number.
A third one was a genuine lookahead bug. A technicals window was bounded to "today" instead of to the scan date it was supposed to reproduce. On a backtest that quietly pulls a bar from the future into a feature that is supposed to be point-in-time. That is the kind of bug that does not hurt you in production at all. It only hurts you in the research you use to decide what to put into production, which is worse, because it corrupts the thing you trust to tell you whether you can trust anything.
The pattern under all thirteen is the same, and it is not specific to options or to finance. Code that invents a plausible value when it should report the absence of one. A null coalesced to a zero. A missing quote coalesced to a range. A date defaulted to now. Each was written by a reasonable person (me) trying to keep a pipeline from blowing up on a missing input. Every one of them traded a loud failure for a silent lie, and silent lies are the expensive kind, because they pass every test you thought to write and corrupt every decision you did not think to recheck.
The fix was not clever. When there is no quote, the spread is now null, and null fails the filter closed. The divergence flip moved ahead of the conviction scores. The window got pinned to the scan date. None of it shipped a new feature. It removed the pipeline's ability to substitute a confident fake for an honest unknown.
What I changed in how I work is smaller and more durable than any one of those patches. I stopped exempting my own data from the suspicion I aim at the model. When a field looks right, that is now a reason to check where it came from, not a reason to move on. The values flowing through your system deserve the same adversarial read you give the language model's output, because in my case they were wrong months before the model ever was, and they were wrong in a way that looked exactly like working.
If you are standing up AI on top of a data pipeline you have never audited this way, that is the work I do. It starts at /services.