Sunday, June 17, 2007

Trade Like a Scientist - Part Three: Three Common Mistakes of Traders

In the first two posts in this series, we examined a scientific mindset and how it affects trading practice. Let's now turn the tables and view three common trading mistakes through the scientific lens:

1) Mistake #1: Trading Without Understanding - Sometimes traders put their capital at risk without taking the time to observe market patterns and integrate these into a concrete explanation of what is happening in the marketplace. A number of traders I work with observed the recent rise in interest rates very early in the move and formulated ideas of shorting rate-sensitive sectors. They tested their understandings with initial positions and scaled into the idea as markets confirmed their views. How different this is from simply putting a position on because a market is making a new high or low!

2) Mistake #2: Oversizing Positions - Many psychological problems in trading can be traced back to excessive position sizing. Traders trade too large for their account size in order to make windfalls, not in order to test their ideas. Scientists conduct many tests before any hypothesis is truly supported, and they test many hypotheses before they accept theories as versions of truth. If you were a lab scientist, would you risk your entire grant funding on a single experiment? Of course not; a single study could fail for a variety of reasons, including experimenter error. Similarly, any single trade or idea can fail for a variety of reasons. A true scientist knows that his or her understanding will always fall short of reality. That is why scientists will conduct doable experiments to refine their ideas before they dedicate significant resources to large investigations.

3) Mistake #3: Not Knowing When You're Wrong - A scientist does not actually test his or her hypotheses. Rather, each experiment is framed as a test of the "null hypothesis": the proposition that variables of interest do *not* affect the outcomes under study. Scientists thus never accept their hypotheses; they at best only reject null hypotheses. Embedded in this perspective is the idea that it is crucial to know when it is necessary to accept that mull hypothesis and conclude that a view is not supported. Can you imagine a qualified scientist becoming emotional because an experiment produces no significant differences and then conducting numerous revenge studies?! Traders, however, sometimes do just that. They don't have rational stop losses identified and so can't terminate their "experiment" at a prudent time. That leads them to take on excessive losses and react out of frustration rather than understanding.

A simple checklist would aid many traders who would become their own performance coaches:

1) What is my understanding of this market and what is the evidence behind it?

2) How much of my capital am I initially willing to devote to my understanding of this market?

3) What outcome(s) would lead me to devote more capital to my idea and what is the maximum portion of my portfolio I'm willing to put at risk on this idea?

4) What outcome(s) would lead me to abandon my idea and how much am I willing to lose on this idea?

Many bad trades could be avoided simply by requiring oneself to answer these questions aloud prior to any trade.


Michael Shopshire said...

Dear Brett,

Great series of posts!!

But in response to your question:

"Can you imagine a qualified scientist becoming emotional because an experiment produces no significant differences and then conducting numerous revenge studies?"

I don't need to imagine it. I have seen NIH funded scientists act emotionally by refusing to admit their theory was not supported (fail to reject the null), and emotionally blame the lab staff and anyone they can find in a futile attempt to "undo" an outcome they cannot accept.

My only point I am making is that scientists are human also, and some are prone to the same emotional thinking in which novice traders engage. When I first read about concepts like thinking in probabilities or taking a loss and moving on, I had wished that I could convince some scientists I knew to think in these more logical ways. Science is even more competitive than trading, and when people are under stress and uncertainty, they tend to act in similar ways.

Brett Steenbarger, Ph.D. said...

Great point, Michael; thanks for the comment. If athletes and traders are not immune from emotional disruptions of performance, scientists probably aren't as well!


Vlad said...

Mr. Steenbarger,

I am afraid I've found all 3 of your posts about a 'scientific approach' to be lacking in substance.

Your description of a scientist as some sort of a detached being impassionately observing empirical data points either confirming or refuting a hypothesis misses the point by quite a bit.

The crucial difference is not whether you get upset when your theory is proven wrong (everyone would be, being human and after putting some work and energy into formulating a theory and collecting data) -- rather, the key is the methodology for how such a proof is obtained.

A scientific approach would be precise about the apriori assumptions being made, the exact model being used (the model is necessarily an imperfect simplification of the real world), and an objective procedure for using observed data to refine the model.

It just so happens that the above approach would frequently lend itself to quantitative/analytical techniques -- but it does not have to be. Being scientific really means being honest and precise about models and model assumption that are verifiable.

I would like to stress the last point again: the overall procedure needs to be objectively verifiable, regardless of whether the process is highly mathematical or not. For example, when we read about Jesse Livermore's adventures in "Reminiscences of a Stock Operator", we can see that, despite many human failings, he had a rational core to his thinking when he sampled a particular market depth with trial buys/cells. On the other hand, the many users of the so called 'technical analysis' systematically fail to achieve reproducible results because they are not clear about assumptions and models behind their seemingly-analytical techniques and hence do not have an objectively verifiable procedure overall.

By itself, the fundamental building blocks of rational inference I am hinting at are nothing new. Physics, machine learning, and even such non-scientific fields as statistics all leverage them. The particular challenges even when a proper scientific approach is used for trading include:

1. if a market model is formulated and even uses probability as the tool for modeling uncertainty, it is valid for a limited amount of time as the underlying probability processes are constantly evolving. Quantifying this duration of validity and how well the model generalizes is not trivial.

2. even the most rigorous models cannot be proven or disproven if the data is insufficient. In fact, this is a major challenge in trading compared to, say, physics where repeatable experiments can be staged at will, more or less.

Some of what you write sounds like you are using single (or very few) data points to corroborate or refute your assumptions. Yet you make no references to how you manage to draw conclusions from such limited data. How many observations do you need to "refute the null hypothesis"? Will the model have run out of its validity time frame by the time you've collected enough such data points? -- these and related questions need to be addressed before you can qualify to recommend a "scientific approach" to your readers.

Otherwise as a scientist myself, I find your recommendations to be misleading at best in their false promises.

pwh's comments on your "stock market correlations" post was a great commentary on how you have managed to fall into this non-scientific trap very recently yourself.

scott said...

Dear Brett,
Good post. But I have to agree with the first poster. As a scientist with 15 years research experience, I can tell you that what you are describing is the "ideal" scientist, not any real scientist I have known. Scientists tend to be very biased towards their hypothesis. Due to the competitiveness of the field, risk taking is favored - not so much the "hedging" you described. It's more like if I gave you $100,000 to trade and said make more than $1,000,000 in the next two years or you get nothing and you're fired.


Brett Steenbarger, Ph.D. said...

Hi Vlad,

Thanks for the critique; I do agree with the gist of your comments. Much of the research I post to the blog is what's known as qualitative research. It can be thought of as hypothesis generating research. I do not post inferential statistics (tests of significance and the like), which would be necessary to take these hypotheses and turn them into scientifically validated trading systems.

The approach I'm taking is more central to many of the social sciences than the physical sciences. But you're right: it would be a mistake to take hypothesis building research and treat the conclusions as established fact.


Brett Steenbarger, Ph.D. said...

Hi Scott,

Good point, and you're right: scientists aren't immune from human frailty. I have to say that, among the million dollar plus grant funded physicians I've worked with, they are passionately dedicated to their work, but also broadly diversified in their interests and studies. I see lots of similarities between successful professional traders and well-published/well-funded principal investigators in academic medicine.