Why Should We Trade Systems?

  • September 19, 2003
  • Dean Hoffman

I am
often amazed at the unbelievable amount of worthless investment
advice offered. The reason I say that is because the vast majority of advice
offered seems to be nothing more than opinions based on little to no real
scientific research. Its similar to the countless “cures” that you can find in
health food stores. Nothing more than untested and unsubstantiated claims.

My opinion on this continues to get stronger and stronger over the years. I have
tested countless methods that the authors purported to be “money makers,” only
to see the vast majority of these ideas being nearly worthless. Many of these
authors are very respected and have well-established followings. In some cases
they have near-celebrity status!

A good example of this is Chart Patterns. There is an entire industry built on
chart patterns. Things such as triangles, pennants, rising wedges, etc,. etc.,
etc. Unfortunately, if you ever try to code these patterns, you will first find
that patterns are extremely subjective. A hundred people looking at the same
chart will see 100 different patterns. In addition, most “obvious” patterns are
only obvious after they have confirmed themselves. This is nothing more than
hindsight bias. When you put the rubber to the road and actually test many of
these “chart patterns,” they don’t hold up at all!

Think about this, you wouldn’t put a powerful drug in your body that was not
first rigorously tested in a scientific manner, would you? Why would you expose
your financial health to ideas that were not also rigorously tested in a
scientific manner? It’s amazing the financial advice people will take based on
the flimsiest of evidence!

What I am suggesting is that the ideas you use should be ones that have gone
through rigorous scientific analysis. Ideas that have been broken apart in
hundred if not thousands of different ways and then tested across thousands of
different examples, to see how they would have performed.

The following text was taken from a popular trading book, Decision Traps,
by J. Edward Russo and Paul J.H. Schoemaker. In this book, nine different types
of decisions were tested using three different decision methods. The accuracy of
the decisions was then compared and analyzed for effectiveness in predicting
final outcomes. The investigator looked at different types of decisions,
predicting grades, predicting recovery from cancer, performance of life
insurance salesmen, as well as predicting changes in stock prices.

He used three different decision-making processes:

1. An Intuitive Prediction Model

2. A Subjective Linear Model

3. An Objective Linear Model.

Decision Models

Intuitive
Prediction Model (Discretionary Trader)

Intuitive prediction is defined as making a
decision without the use of any objective or quantifiable data. For instance, in
trying to predict the academic performance of graduate students, the researches
asked their advisors to do so without seeing their grades and just by talking to
them. The decision-makers had to rely on their intuitive impressions and any
other factors they thought relevant (how the student dressed, their language
skills, grooming habits, etc.). This is the same way discretionary traders make
trading decisions – using intuition and gut instinct. In predicting the stock
prices, it is highly likely that the researcher engaged a discretionary trader
to predict the future prices of stocks.

Subjective Linear
Model (Technical Trader)

A Subjective Linear Model is a much more complex
decision making process. It starts with the interviewing experts in a field and
learning how they make decisions. The researcher literally asks the expert how
he or she makes decisions and they respond by explaining how they make their
predictions. Although these experts are not using quantifiable data, they have
enough experience and knowledge in their field to be successful. This decision
making process is then outlined by the researcher.

For instance, a physician, highly experienced in
treating cancer, has probably become fairly adept at predicting the life
expectancy of his patients, even without using any objective data. The
researcher interviewed the physician and attempted to determine exactly how the
physician made this assessment. Then the researcher put this newly quantified
data into a regression model and attempted to predict the life expectancy of
cancer patients.

This is very similar to how a technical trader
makes decisions. He goes to seminars and reads books to learn how the experts
make decisions using technical indicators. He then takes what he learns and
attempts to trade like he experts. In a sense, he does his own regression model
of the expert’s process to make trading decisions.

Objective Linear
Model (System Trader)

For the Objective Linear Model, the researcher
developed an objective model based on historical tests and observations to
predict results. This is defining and using quantifiable data, running
historical tests, and then using the results of the tests to predict future
outcomes.

For instance, the researcher would look at reams
of physical data from terminal ill patients, and correlate the data with how
long the patient lived. After running the historical tests, the researcher would
then obtain the physical data form cancer patient, and using the historical test
data, attempt to predict how long that cancer patient will live.

This is exactly what a system trader does. He
runs historical tests and then uses that data to take a position in the market.
He uses objective quantifiable data tested historically to make his trading
decisions. The following table shows the results of tests.


image src=”https://tradingmarkets.com/media/2003/Hoffman/dh091903-01.gif” width=”420″ height=”463″ />

In every case, the Subjective Linear Model
outperformed the Intuitive Prediction Model but only by a small margin. If you
look at predicting the changes in stock prices, the Subjective Linear Model only
slightly outperformed the Intuitive Prediction Model.

The real insight from this study comes when we
look at the results of the Objective Linear Model. In every case, the Objective
Linear Model outperformed both the Intuitive Prediction model and the Subjective
Linear Model. In some cases, the improvement was minor, and in others it was
substantial. It is interesting to observe that the greatest improvement came
when using the Objective Linear Model in predicting the changes in stock prices!

Hopefully I have caused you to
think a little but about using thoroughly researched systematic methods in your
trading (if your not already). My nearly 12 years of computerized research has
given me the opinion that roughly 90% of the methods in the public domain
(books, seminars etc.) has little to no real value.

Unfortunately, many charlatans
and snake oil salesmen have used the guise of computerized research to sell
equally worthless creations. As they say “if it was so easy, everybody would be
doing it”.

The real key is learning how to
use the power of computerized research to develop real tradable systems without
falling into the many possible traps including optimization (discussed in a
previous article).

In future articles, I will be
discussing what I believe to be the correct ways to actually build tradable
non-optimized systems.

Dean Hoffman

P.S. Please feel free to email
me with your questions at:
deanh@tradingmarkets.com