Walk-Forward, or You’re Lying to Yourself

Most backtest disasters are not exotic. They are a small set of mechanical leaks that let tomorrow’s information into yesterday’s decision, and each one inflates results in a way that feels like skill. Here is the harness we use to keep a test honest — and the specific artifacts each rule exists to prevent.

The five rules

• Measure what you trade. Score returns or relative returns, never the price level — predicting the level yields an R² near 0.997 from pure autocorrelation, not skill.
• Walk forward. At each step, train only on data available up to that point and predict the next non-overlapping period; the window expands as time moves on. Never randomly shuffle time-series data into train/test.
• No scaler leakage. Normalize features (z-score) within each date’s cross-section only, fit on that day alone — never across the whole history at once.
• Point-in-time fundamentals. Key every reported number to its filing date and merge it onto the price calendar as-of, so the model only ever sees what was public.
• Benchmark and decompose. Compare against a naive baseline, and break any apparent edge into factor exposures before believing it.

Why shuffling is fatal

Randomly splitting time-series rows into training and test sets is the most common silent killer. It places days from the future into the training set used to predict the past, so the model “learns” from events that had not happened yet. The backtest looks excellent and the live strategy fails immediately, because in production the future is genuinely unavailable. Time-series data must always be split by time, never at random.

The subtle one: cross-sectional leakage

Normalizing a feature across the entire history — subtracting the all-time mean, dividing by the all-time standard deviation — quietly leaks the future’s distribution into every past observation. The fix is to z-score within each date, using only that day’s cross-section of names. And fundamentals must be keyed to when they became public: a company’s quarter-end numbers are filed weeks later, so keying them to the quarter they describe hands the model information before the market had it.

About this series: every figure comes from a leak-free research harness on US equities — point-in-time index membership, fundamentals keyed to filing date, expanding-window walk-forward, and transaction costs charged. Statistics are gross and in-sample unless noted, and describe published anomalies, not a Yayati product. Standing caveats: roughly a third of true historical index members are unpriced by the naive data source (survivorship); a 2 bps cost assumption is optimistic; fundamentals are post-2009 XBRL.

This article is for educational and informational purposes only and is not investment, tax, or legal advice. It describes findings from an internal research program about publicly documented market anomalies and research methodology; it is not a description of any Yayati product or its results. Research statistics are gross, in-sample illustrations subject to survivorship, data-coverage, transaction-cost, and modeling limitations described in the text, and do not represent actual trading or any client account. Past performance and backtested results are not indicative of future results. Yayati Asset Management is a Registered Investment Adviser. © Yayati Asset Management. VOLT™ and PLASMA™ are trademarks of Yayati.