Many successful traders share one habit – they backtest their trading strategies. Backtesting your trading strategy will not alone guarantee that you will become profitable, but it is a giant step in the right direction. In this article we examine some potential biases that can creep into your backtesting, and we will look at how to minimize the impact of these biases.
There are many problems that can occur when you backtest your trading system, but most problems fall into one of three categories: postdictive errors, too many variables, or failing to anticipate drastic changes in the market. Each of these errors is explained, along with methods of avoiding errors.
Click here to learn how to utilize Bollinger Bands with a quantified, structured approach to increase your trading edges and secure greater gains with Trading with Bollinger Bands® – A Quantified Guide.
1. Postdictive Error
The postdictive error is just a fancy way of saying that you have used information only available “after the fact” to test your system. Believe it or not, this is a very common error when testing trading systems.
This error is easy to make. Some software will allow you to use today’s data in testing a trading system, which is always a postdictive error (we do not know if today’s data is useful yet for predicting the future, but we certainly do know if it is useful in predicting the past). Wouldn’t you love to be able to use the closing price of the GBP/USD to predict what the market will do today? Of course you would, I definitely would, but unfortunately, this information is not available to us until the day is over. For example, you may have a system that incorporates the closing price, then this obviously means that the trade cannot be initiated until the day is over, otherwise this is a postdictive error. Another example may help illustrate the postdictive error, if you have a rule in your trading system about highest prices, then you will have a postdictive error. This is because highest prices are often defined by data that comes later, in the future.
The way to avoid the postdictive error is to make sure that when you backtest a system that only information that is available in the past at that point in time is used in backtesting. With manual backtesting or backtesting with forex tester you can accomplish this quite easily, but with automated backtesting the postdictive error can sneak into your trading system.
2. Too Many Variables
This is also known as the “Degrees of Freedom” bias. This simply means that you have too many variables, or trading indicators in your trading system. It is very possible to come up with a trading system that can explain past price behavior of a currency pair. In fact, the more indicators you add, the easier it often becomes. The problem arrives when you want to apply this system to the future.
Often when a trading system has too many indicators it can predict the behavior of the market during a time period extremely well. But, that’s all the system is good for, because in the future the system falls apart.
The above statement is often difficult for traders to come to grips with, but it is true. Consider what William Eckhardt, of the New Market Wizards has to say about trading systems, In general, the delicate tests that statisticians use to squeeze significance out of marginal data have no place in trading. We need blunt statistical instruments, robust techniques.
Obviously, he is warning against the degrees of freedom error and suggesting that simple trading systems are more likely to stand test of time. This is absolutely true.
Some of the most powerful trading systems available are extremely simple.
Keep this in mind as you trade, and as you attempt to find a profitable trading system. Most traders will find that with experience, they become more likely to embrace the view that simpler trading is preferred over a complex approach.
3. Drastic Changes in the Market
Many traders forget to anticipate unforeseen events that will occur in the future. It doesn’t really matter that you don’t know what is going to happen in the future – because you do know this: there will be times in the future when the markets will behave erratically. When this happens, you should have designed your trading system to remain functioning during these times.
Perhaps some examples may help with this: When Saddam Hussein was found (over the weekend), the currency markets reacted quite drastically on Monday’s opening. When the global financial crisis started unfolding in September 2008, most currency pairs traded with much more volatility than had been seen for years.
The fact is that there will be unexpected events in the future, and these events will affect the markets, so the best thing you can do is to be prepared. How do you prepare for the unexpected? Consider these simple solutions:
1) Exaggerate your expected losses. If your backtesting reveals a maximum loss of $5000, assume a maximum loss of $10,000. Will your trading systems still be profitable under these conditions?
2) Decide on an appropriate level of risk for each trade. Remember that even this level of risk is likely to be exceeded. If you have decided to risk 1% on each trade, you should assume that sometime in the future, you may be in a trade and an unexpected event will occur, and your trade will not lose 1%, but instead 5% will be lost.
3) You should have a contingency plan set up. That is, how will you exit a trade if something bad happens and you cannot access your account? For instance, what happens if your trading platform is inaccessible and you desperately want out of a trade? Most brokers offer a telephone line to traders for these instances. Do you have the phone number?
4) Do you have a maximum risk level set? This would be applicable if you have several trades open simultaneously. If you decide to risk 1% per trade and you have 7 trades open simultaneously, does this mean that you will be risking 7% of your account? Or have you decided on a maximum risk level of say, 3%? Keeping in mind that the unexpected will occur, you should probably have a maximum risk level for those times when you have several open trades.
5) What is the maximum drawdown (amount of money your trading system loses over an extended period of time) you are willing to tolerate? Keeping in mind that you (and you are not alone) are more likely to overestimate the severity of drawdowns that you can withstand, it is important to be realistic. If you lose 30% of your account will you stop trading? What about if you lose 50%? Or if you see 70% of your account disappear? Again, the best way to plan for drawdowns is to do extensive backtesting to find out what sort of historical drawdowns your trading system experiences and then plan for even worse drawdowns in the future.
Anticipating drastic changes in the markets is the single best way to preserve the equity in your account.
So, you know that successful traders share this habit – they backtest their trading strategies. You know that backtesting separates the wealthy traders from those who lose money. You also know several ways of incorporating backtesting into your trading regimen. And you know of the pitfalls – what to look out for – when you are backtesting, so that you can get the most out of the process. But, what exactly, will you get out of backtesting your trading system? In the next article I will explore the side effects of backtesting.
Walter Peters, PhD is a professional forex trader and money manager for a private forex fund. In addition, Walter is the co-founder of Fxjake.com, a resource for forex traders. Walter loves to hear from other traders, he can be reached by email at firstname.lastname@example.org.