The purpose of this article is to make the strongest case possible for back testing as a crucially important way of understanding your system. In other articles I will suggest that too much back testing is bad and that you can learn too many wrong lessons if you’re not careful. That said however, back testing is an essential part of a complete trading plan.
Back testing can take many forms. In some cases an experienced trader who is considering an idea that is similar to previously reliable systems may only need a minimum of back testing to be convinced that idea is worth trading with live money at a reduced risk level. There is synergy in professionalism and experience that should not be neglected or underestimated.
But even an experienced trader should carefully consider the results of a detailed backtest when taking on a new strategy or operating in a new time frame in order not to be misled by the constraints of his own experience.
Some people are not convinced by the quality of an idea unless they see it work over multiple time frames, in multiple markets and in all different market conditions. Others are satisfied that an idea only has to work within the definable set of parameters to be tradable.
This is a matter of personal taste since it comes down to a personal risk of capital rather than an academic exercise in the pursuit of absolute truth. Leave that for the academics. We want to make money as traders.
Properly constructed, back testing will identify whether or not this idea has a persistent edge, and under what conditions it will manifest. By properly controlling for different parameters we can isolate those which add the most value to this particular proposition. We can test for robustness and see how sensitive the edge is to changing parameters.
We may be able to identify specific market conditions where the edge is significant and tradable. We may be able to identify a subset of the total market trading targets in which this idea works best.
Back testing should tell us what the win rate percentage is likely to be, the importance of slippage and commissions, the trading frequency, the maximum adverse excursion, the longest normal winning and losing streaks, and both the maximum and average wins and losses.
One of the most important result sets for analysis is the distribution of results in the form of a frequency histogram. We would like to see a somewhat normal distribution that has most of the trades clustered around the mean and with an orderly profit tail to the right that suggests we have the possibility of large winning trades. We would also like to see a carefully controlled left tail of losses which suggests that we are able to engineer our risk carefully.
Having this kind of data in hand allows us to determine where, when and under what conditions this idea is tradable and what the expected results should be. When we proceed into live market trading as a prototype system with much reduced real risk, we can then compare actual results with live money to backtest results to see if the trade can be managed as intended.
Under these kinds of conditions and looking for this kind of information, back testing is an important part of the traders’ repertoire.