Why You Should Run A ‘Worst-Case Analysis’ Report
Many
system test reports or real time trading statements only show
you the performance based on a single starting date. This can be an extremely
limited view because that single date could have been “cherry picked”
as the optimal historical starting date. The date shown could have been right
in front of a series of winning trades. This could give you the wrong impression
about how much capital you might have needed to trade the account, had you started
on a different date.
What if you had started
on the worst possible date in history and had a series of 20 losing trades in
a row? It’s possible that you would have needed two or three or 10 times the
capital to trade the system as the original test or real time account might
have shown! This also means the return on invested capital could have been two
or three or 10 times LESS than reported from a
different starting date. This is much more common than many traders realize
and would not have been seen with a single starting date (either hypothetical
or real time).
The best way to deal with
this in our opinion is to run a “worst-case analysis“ report. What
this report does is tests the system from every single starting date in the
series. In other words, if over the last 10 years the system had 5000 trades,
then this test would re-test EVERY single trade
as having been the trade you started with. This allows you to see the results
had you started on the worst possible day in history. You
can also use this data to run a frequency distribution and see things like average
drawdowns, average first year returns, average margin requirements, etc.
Once again, if you only
look at this in the typical single starting date format you can miss seeing
all the potential risks. The worst-case reports turn a two-dimensional single
starting date report into a full color three-dimensional view. In our opinion,
this is the REAL litmus test for hypothetical results.
Quite frankly, we don’t know how you can honestly evaluate a trading system
without it.
Imagine the knowledge you
will gain by seeing system results when run from 5000 different starting points,
rather than just one.:
How many times out of 5000
did it have a losing first year? Second year? How many times out of 5000 did
the first-year performance match the historical average? How many times out
of 5000 did equity reach $1 million by year five? What was the average start
trade drawdown out of 5000 starts? What was the highest? How many times out
of 5000 did drawdown exceed 50%?
In our opinion, answers
to these types of questions will let you know with a much better degree of probability
what to expect from your systems in real trading. This is an excellent tool
for better projecting how your system might perform in the real world.
If you have any questions
or comments on this article, feel free to email me.
dhoffman@traderstech.net