\( \newcommand{\bm}[1]{\boldsymbol{#1}} \newcommand{\textm}[1]{\textsf{#1}} \def\T{{\mkern-2mu\raise-1mu\mathsf{T}}} \newcommand{\R}{\mathbb{R}} % real numbers \newcommand{\E}{{\rm I\kern-.2em E}} \newcommand{\w}{\bm{w}} % bold w \newcommand{\bmu}{\bm{\mu}} % bold mu \newcommand{\bSigma}{\bm{\Sigma}} % bold mu \newcommand{\bigO}{O} %\mathcal{O} \renewcommand{\d}[1]{\operatorname{d}\!{#1}} \)

8.6 Summary

Backtesting of portfolios is an essential part in the process of strategy development and evaluation. Nevertheless, it remains widely misunderstood and the dangers are routinely underestimated. Some of the key points to keep in mind include the following:

• There are multiple reasons why backtest results have to be taken with “a grain of salt,” namely, survivorship bias, look-ahead bias, storytelling bias, overfitting or data snooping bias, turnover and transaction cost, outliers, and asymmetric pattern and shorting cost, among others.

• Arguably, the single main reason why any backtest may be faulty and misleading is overfitting.

• Due to these potential pitfalls, one cannot trust any backtest results provided in academic publications, blogs, investment fund’s brochures, etc.

• Armed with this knowledge, it is recommended to conduct multiple randomized backtests, perhaps combined with stress tests under different scenarios.

The reader has been warned, provided with tools, and hopefully this will serve as a guide for the future.