How to Build a Trading Strategy – Part 4
Matt Radtke is Senior Researcher for Connors Research. Mr. Radtke graduated magna cum laude from Michigan State University with a degree in computer science. He has 25 years of software development experience in companies large and small, including Hewlett-Packard and Bell Northern Research.
Mr. Radtke has been actively trading stocks, ETFs, and options since 2008. Over the past several years he has become increasingly involved with the Connors Group family of companies, first as a student, then as a member of Chairman’s Club, and finally as a consultant, researcher, and author.
Back-Testing
Everyone who’s ever placed a trade, made an investment, worked with a financial advisor, filled out paperwork for a brokerage, or watched more than 15 minutes of CNBC is familiar with the standard investing disclaimer:
Past performance is not a guarantee of future results!
While it is true that the future is unpredictable, that doesn’t mean that history is useless to us. A well run back-test allows us to evaluate how a strategy has performed in the past, and to make some judgments regarding the relative performance of different strategies or even different variations within a single strategy. Just because Strategy X returned an average of 15% annually over the past 10 years doesn’t mean it will produce a 15% gain this year. However, it is quite likely to perform better than Strategy Y which lost an average of 9% annually over the past 10 years.
Do you want to learn the essential skills, techniques, and strategies of high-probability, quantified Swing Trading? Click here to register for The 2013 Swing Trading College. Space is limited so reserve your spot today before it’s too late!
There are a few rules which can maximize the validity of back-testing. These are all aimed at making the back-test mimic future trading scenarios as closely as possible, both in terms of execution mechanics and the environment in which you would be operating.
- Test against a universe that is as similar as possible to the universe that you intend to trade against. We discussed this in the previous section.
- Use a date range that is as long as practical, which usually means the longest period for which you have high-quality data. In some cases, the testing period may be limited by the universe itself. For example, leveraged ETFs started coming into existence in 2006 and 2007, so trying to back-test a leveraged ETF strategy from 2001-2005 would not be productive.
It is particularly important to have your back-test cover a variety of market conditions, including bull , bear, and sideways markets. You never know when the next bull run or crash is just around the corner.
- Eliminate survivor bias. Every year, many stocks stop trading and are delisted, or removed from the exchange. In many cases this is due to merging with or being acquired by another company, or because the company went bankrupt. If your testing universe does not include these delisted stocks, then you are skewing your results toward the “survivors”. If your strategy would have had you entering trades on Enron or WorldCom just before their demise, you would probably want to reflect those losses in your results.
- Use “As Traded” prices and volumes. Most data sources automatically adjust both prices and volumes to eliminate the effect of stock splits, and some also adjust for dividends that the company pays. Let’s say you own 100 shares of company XYZ, and yesterday it was trading at $50/share. Today, XYZ undergoes a 2-for-1 stock split. Your broker will show that you now own 200 shares priced at $25/share. However, if you look at a price chart, yesterday’s price will no longer reflect the $50 share price, but rather will be adjust down to $25 share ($50 divided by 2). Volume will be adjusted in the opposite direction (i.e. increased by a factor of 2) so that total price volume remains unchanged for the day.
Generally, it is helpful to look at split-adjusted prices. If you were looking at a chart (or running an analysis) and the price of XYZ got cut in half from one day to the next, you would think something went drastically wrong unless you also had knowledge of the stock split. For back-testing, however, if our universe is defined by price and/or volume then we want to use the values for those parameters that were actually in effect on the day that we would have entered the trade. If your data provider makes “as traded” values available to you, then you should use them when determining whether a stock qualifies for your defined universe. For all other analysis, use the adjusted price to avoid having indicators return abnormal values.
- Only use data in the back-test that would be available to you as a trader. This may seem obvious, but it is easy to make small coding mistakes which allow your back-test to “cheat”. For example, let’s say that you tell your back-testing system to enter a long trade today on a 3% limit order, but only if today’s RSI(2) value is below 20. The problem is that if you enter on a limit order today, that order will be filled (or not) at some unknown time between today’s open and today’s close. However, you won’t know today’s RSI(2) value until the market closes, so by considering today’s RSI(2) value in the back-test, you’re essentially looking ahead into the future.
This is the reason that we had to express our entry rules with a setup component and a buy/short component. We can only know the 21-day moving average of the volume on the day before we enter the trade. We won’t know today’s MA(Vol, 21) until the end of the trading day, long after we have entered the position.
- Only test scenarios that you are able to execute in live trading. For example, our central thesis requires us to enter trades at the open, and this is very easy to do from within the back-testing environment. As a trader, however, you need to decide if you have a mechanism for doing this. If you have very few trade signals, it may be sufficient to manually place trades when the market opens each day. If you have a larger number of trade signals and your broker supports it, you may need to place the trades in advance using Limit On Order (LOO) entries. If your broker does not offer LOO functionality, then you may be able to get by with a Good Til Canceled (GTC) order that automatically gets cancelled a couple of minutes after the market opens.
With those rules in mind, we can now do an initial test of our gap strategy. There are many tools available for executing back-tests, some of which are free and others which are not. At Connors Research we have found AmiBroker to be an excellent application for back-testing as it makes many testing tasks relatively easy, has a decent array of built-in indicators, and is also extendable with a C-like language called AmiBroker Formula Language (AFL). Even more important than your choice of testing applications, however, is your source for price data. We currently buy data from Commodity Systems Inc., often referred to as CSI. In addition to adhering to the rules listed above, make sure that the data from your provider is easily imported into your selected testing application.
One of the features of AmiBroker is that it allows us to run an optimization, which is a series of back-tests. Each back-test has a unique set of input parameters, which we refer to as a variation of the strategy. So far the only strategy rule that lends itself to using a range of input values is the size of the gap. Therefore, we will begin by varying the gap size from 0.5% to 10% in increments of 0.5%. We will look at the results of the long strategy and the short strategy separately so that we don’t mask any potentially important details.
Long Trade Results
|
# Trades |
Avg |
Win % |
Gap % |
|
73489 |
0.21 |
51.42 |
0.5 |
|
44968 |
0.33 |
52.25 |
1.0 |
|
29671 |
0.47 |
53.23 |
1.5 |
|
20613 |
0.64 |
54.43 |
2.0 |
|
14862 |
0.76 |
54.99 |
2.5 |
|
11025 |
0.91 |
55.43 |
3.0 |
|
8375 |
1.05 |
56.01 |
3.5 |
|
6525 |
1.13 |
56.15 |
4.0 |
|
5245 |
1.24 |
55.75 |
4.5 |
|
4292 |
1.29 |
55.66 |
5.0 |
|
3590 |
1.28 |
55.32 |
5.5 |
|
2985 |
1.30 |
54.57 |
6.0 |
|
2549 |
1.34 |
54.49 |
6.5 |
|
2183 |
1.26 |
53.60 |
7.0 |
|
1877 |
1.15 |
52.32 |
7.5 |
|
1659 |
1.03 |
51.78 |
8.0 |
|
1463 |
0.88 |
50.99 |
8.5 |
|
1319 |
0.83 |
50.72 |
9.0 |
|
1176 |
0.84 |
50.68 |
9.5 |
|
1045 |
0.67 |
50.62 |
10.0 |
Here is an explanation of each column.
# Trades is the number of times this variation triggered from January 1, 2001 – December 31, 2012.
Average % P/L is the average profit or loss for all trades generated by this variation, including the losing trades, expressed as a percentage.
Win % is the percentage of trades which closed out at a profit.
Gap % is the minimum required difference between the low price on the setup day and the open price on the entry day, expressed as a percentage of the setup day low price. For example, if the lowest price on the setup day was $20.00 and the variation requires a 3% gap, then the next day’s open would need to be less than $20.00 – ($20.00 * 3%) = $20.00 – $0.60 = $19.40. The more expedient way to arrive at this threshold is to simply multiply the low price ($20.00 in our example) by 97% (100% – 3%).
What can we learn from our initial test results? Let’s look at the columns in order.
Over a 12-year period, the number of trades ranges from over 73,000 down to 1045. Even at the low end, that means we’re generating almost 100 trades per year, or about 2 per week on average. Your own trading style will dictate how many trades are “enough” for you.
The Average & P/L, sometimes called the Average Gain, starts at a rather meager 0.21%, increases steadily to around 1.3%, and then tails off again. Two things stand out: the middle of the % Gap range that we tested appears to be the sweet spot, and there is really no point in testing Gap Percentages in 0.5% increments. Testing in 1% increments will reduce our test time without losing and crucial information.
The Win %, or Win Rate, is fairly steady in the 50% – 55% range. For many day traders, this is perfectly acceptable. Those who hold their trades for multiple days typically look for something a bit higher. Of course, bigger is always better in this category, so as we go on we will look for opportunities to increase our success rate.
Now let’s look at the results on the short side:
Short Trade Results
|
# Trades |
Avg |
Win % |
Gap % |
|
81133 |
0.16 |
51.98 |
0.5 |
|
46862 |
0.19 |
52.15 |
1.0 |
|
29777 |
0.23 |
52.37 |
1.5 |
|
20261 |
0.28 |
53.12 |
2.0 |
|
14458 |
0.35 |
53.60 |
2.5 |
|
10626 |
0.39 |
53.53 |
3.0 |
|
8038 |
0.45 |
54.18 |
3.5 |
|
6297 |
0.46 |
54.04 |
4.0 |
|
5001 |
0.51 |
54.85 |
4.5 |
|
4063 |
0.61 |
55.28 |
5.0 |
|
3373 |
0.59 |
55.23 |
5.5 |
|
2807 |
0.66 |
55.54 |
6.0 |
|
2341 |
0.76 |
56.04 |
6.5 |
|
1977 |
0.79 |
55.79 |
7.0 |
|
1690 |
0.82 |
56.15 |
7.5 |
|
1434 |
0.83 |
56.69 |
8.0 |
|
1261 |
0.90 |
56.62 |
8.5 |
|
1107 |
0.88 |
56.64 |
9.0 |
|
988 |
0.97 |
56.88 |
9.5 |
|
871 |
0.90 |
56.49 |
10.0 |
The columns are the same as in the previous table, except that for the short strategy the Gap % is the minimum required difference between the open price on the entry day and the high price on the setup day, expressed as a percentage of the setup day high price. In other words, we enter when today’s open is greater than Setup High x (100% + Gap %).
The Short Trade statistics follow a pattern similar to the Long Trades. One difference is that the “sweet spot” for the Gap % (as measured by Avg % P/L) seems to be more toward the upper end of the range. Therefore, we’ll keep the entire range of Gap % values in place for our testing, but we’ll increment the values by 1% instead of 0.5% for efficiency.
Now that we have some base results to work from, we can start to refine our strategy. The next two installments of this series will examine how additional entry and exit rules can improve our results.
Click here to read How to Build a Trading Strategy – Part 1
Click here to read How to Build a Trading Strategy – Part 2
Click here to read How to Build a Trading Strategy – Part 3