When Optimizing, Make Sure You Avoid This
To the
new systems developer, one of the most exciting things to play with
is optimization. Optimization is using the power of the computer to examine
every possible sequence of parameters and rules, to find those that have worked
out best in the past. With enough computer crunching power, it’s possible to
find systems that perfectly “predicted†the past. Many traders then think they
have discovered the holy grail of trading systems and jump into the markets with
their new super-predictive algorithms, only to find they fall apart in real
trading!
“What happened?†they ask
themselves. The answer is that they have created a system that was curve fit.
Curve fitting is where a system has been optimized to a unique set of historical
data. The problem is that the markets will behave much differently in the future
than the past, therefore, a “perfect†trading system in the past could be
useless in the future.
For example, let’s assume we
wanted to optimize a set of nickels that only landed on heads. What we could do
is flip a million nickels and only select those that landed on heads. Then, we
can take those remaining nickels and flip them again, once again only choosing
those that land on heads. We could repeat this process over and over again, each
time only choosing those nickels that land on heads. At this point we might
conclude that we had narrowed down our nickels to those that were “optimized†to
land on heads. We could then go out and make large bets with those nickels,
putting all our money on heads. We would quickly make a fortune right? WRONG!
Unfortunately, we would very
quickly lose our money. These nickels were not optimized for heads; they always
did and always will have 50/50 odds. What confused us is that we thought that we
had found a predictable set of nickels, when in fact we had just found a
statistical coincidence!
These kinds of errors find
their way into trading systems all of the time. When developing a system, it’s
imperative that optimizing is avoided as much as possible. Robust methods need
to be developed. There can be a place for certain types of optimizing, but it
must be handled correctly. In my next article I will talk about some of the
things that need to be done to avoid optimizing and develop robust non “curve
fit†strategies.
Please feel free to email your questions/comments to me at deanh@tradingmarkets.com.