You Can’t Backtest a Number That Gets Rewritten
Key takeaways
- Some data providers overwrite past values as estimates and ratings update (restatement).
- Backtesting today’s restated snapshot against historical prices is look-ahead bias.
- You are grading the model with information that did not exist at decision time.
- Proprietary daily-restated ratings (IBD/Zacks-style) are especially prone to this.
- The only honest fix: archive the live signal forward and build your own point-in-time history.
Look-ahead bias is not always a careless date error. Its most seductive form hides inside data that gets rewritten — estimate-revision feeds and proprietary ranking systems that quietly update their own historical values as reality unfolds. Backtest against that, and you are testing a strategy that knew the future.
Where exactly is the cheat?
Suppose a vendor restates last quarter’s rank for a stock once this quarter’s earnings arrive. Today’s database now reflects information that did not exist back then. Backtest that snapshot against quarter-ago prices and the strategy appears to “predict” moves it could only know in hindsight. The equity curve looks strong for the worst possible reason — it is reading tomorrow’s newspaper. Daily-restated rating systems are the classic offender: the rank you can download today is not the rank you would have seen on the historical date.
How do you test these signals honestly?
Point-in-time data: capture the signal as it stood on each date and never overwrite it. Where the underlying data exists in a clean archive — SEC filings keyed to their filing date, for instance — you can reconstruct it. Where a vendor only publishes a restated snapshot, there is no backfill that restores integrity; the only remedy is to start recording the live feed forward and build your own point-in-time panel over time.
That is exactly what we did with one famous proprietary system. Rather than backtest its restated ratings, we rebuilt the underlying idea from scratch out of point-in-time components — momentum, an earnings-surprise measure keyed to filing date, and a profitability measure — so every input reflected only what was knowable at the time. It is slower, and you cannot manufacture history you failed to record. It is also the difference between a real test and a flattering illusion.
The discipline
If a data field can be edited after the fact, assume it has been — and never trust a backtest built on it until you have reconstructed it point-in-time or archived it forward yourself.
About this series: every figure comes from a leak-free research harness on US equities — point-in-time index membership, fundamentals keyed to filing date, expanding-window walk-forward, and transaction costs charged. Statistics are gross and in-sample unless noted, and describe published anomalies, not a Yayati product. Standing caveats: roughly a third of true historical index members are unpriced by the naive data source (survivorship); a 2 bps cost assumption is optimistic; fundamentals are post-2009 XBRL.
This article is for educational and informational purposes only and is not investment, tax, or legal advice. It describes findings from an internal research program about publicly documented market anomalies and research methodology; it is not a description of any Yayati product or its results. Research statistics are gross, in-sample illustrations subject to survivorship, data-coverage, transaction-cost, and modeling limitations described in the text, and do not represent actual trading or any client account. Past performance and backtested results are not indicative of future results. Yayati Asset Management is a Registered Investment Adviser. © Yayati Asset Management. VOLT™ and PLASMA™ are trademarks of Yayati.
See VOLT™ on a real position.
The tax-smart option overlay behind this paper, for concentrated stock.