Thursday, September 14, 2006

The Most Common Mistake Traders Make

What is the trader's most common mistake? Candidates abound: failing to diversify capital, adding to losing positions, trading through stop-loss levels, trading on hot tips and news items, revenge trading to recover losses, overtrading during slow periods: the list goes on and on.

I'd like to suggest, however, an even more fundamental error that gets traders into hot water: They confuse descriptive statistics with inferential ones.

Allow me to explain.

A descriptive statistic summarizes some characteristic of a sample of a population. Lets say I go out on the sidewalk and survey passers-by as to their political leanings. I find that 65% of the people in my survey support the mayor and plan to vote for him. That is a descriptive statistic. If I conduct the survey several times in a row and find that the percentage stays steady at 65%, this, too, provides a description of my sample.

Much of the information that traders work with is descriptive in nature. Consider these statements:

* The market trendline is up;
* Advancing stocks are ahead of declining stocks by 2:1;
* We are making new weekly highs in the market;
* Volatility is at a new monthly low;
* The market made a breakout from the trading range.

All of these are descriptive. They take a sample from a price or indicator series and describe characteristics of that sample.

All good science begins with observation and description. Such qualitative, descriptive analysis allows us to notice regularities--or patterns--in our data and frame meaningful hypotheses based upon these.

Descriptive statistics can lead us to the formulation of hypotheses, but they cannot provide tests of those hypotheses. That is the role of inferential statistics. To test a hypothesis, we must evaluate multiple samples and verify the existence of suspected patterns. My hypothesis, for instance, in the above example might be that the mayor will win the upcoming election. To test that hypothesis, however, I would have to conduct multiple surveys in different neighborhoods at different times.

It would be a mistake to assume that a description of a sample necessarily reflects the properties of the entire population. The sample of voters on my sidewalk may not reflect the composition of voters in the entire city. Generalizing from a single sample to an entire population is dangerous: it confuses a hypothesis with a conclusion.

But that is the mistake that many traders make: They proceed directly from descriptions of recent markets to assumptions about future ones. They assume that the sample of recent price changes can be generalized to the population of all price changes. Thus, they'll conclude that the market is going to rise because the trendline is rising; that we have a bull market because stocks have been advancing.

The only way to know that, however, is to test a number of rising periods and see whether indeed rising prices in the past lead to future rises significantly more often than not. That's the role of inferential statistics, such as tests of significance.

Jeffrey Miller, Ph.D., in his blog A Dash of Insight, neatly frames the issue: We need ways of deciding whether or not market moves (or trader P/L) are due to luck.

That is the most common mistake traders make. We assume that strings of events are meaningful, when--much of the time--they reflect the luck of chance. The patterns perceived by our minds are not necessarily patterns that exist in nature.

The quantitative, system trader trades patterns that have been tested with inferential statistics. The discretionary trader trades descriptive hypotheses that he/she validates with updated, real-time readings of market conditions. Is the discretionary trader justified in doing that? The same inferential tests that inform us of the validity of trading systems, when applied to the trader's trading results, will answer that question.

That is why I find programs that establish and evaluate trader performance metrics, such as Trader DNA, to be so important. Only by evaluating results can discretionary traders know whether or not their judgments are adding skill to luck. Of the discretionary traders I know personally who have made more than a million dollars a year for multiple years, one characteristic stands out: they obsessively keep score. They track their results carefully, figure out what they're doing right or wrong, and make periodic adjustments. As Bill Rempel perceptively noted, they, too, are trading systems.


Sabretache said...


I'll say this. Your stuff is uniformly thought provoking. Nonetheless There's a lot in that post I can't go along with. The notion of science (strictly defined) being applicable to a thorough understanding of market behaviour for example. As I posted before, with price movements being a function of human decision-making in what I suggest is a difficult probabalistic distribution, any such 'science' is at best analagous to the social sciences - and look where they're leading us!.

Your observations on randomness are on the button though. Don't know if you've read NN Taleb's "Fooled by Randomness". If not I heartily recommend it. Here's what he has to say about 'survivorship bias' to the relevance of trader past performance:

"The information that a trader made money in the past, just by itself, is neither meaninful nor relevant. We need to know the size of the population from which he came. In other words, without knowing how many traders out there have tried and failed, we will not be able to assess the validity of the track record. If the initial population includes 10 traders I would give the performer half my savings to trade without a blink; If it comprises 10,000 I would ignore it."

In other words give a monkey a word processor and eventually he will produce a key sequence amounting to sublime poetry. Those that succeed by pure luck continue to trade with the rest - at least until their luck runs out. Also, the problem with successful traders (myself included) is that they have big egos and are thus inclined to ascribe their success to their own ability - until the humbling black swan hits that is - when by definition they are no longer successful and it becomes all somebody else's fault.

Beware any notion of 'normal distributions' in the markets I say. :-))

Brett Steenbarger, Ph.D. said...

Thanks for the recommendation of the Taleb book; it is, indeed, must reading on this topic. And I wholly agree: assuming normal distributions of market outcomes is a quick road to the poor house once we get those multi-sigma runs. I appreciate your comment about the blog being thought provoking. That's my purpose: to help traders question and think hard about what they're doing. A guru offers answers for followers; a teacher raises questions for inquiry. I try my best to bet the latter.


James said...


Are there any trading simulation software programs (affordable ones) that you recommend?

the best to you,


Brett Steenbarger, Ph.D. said...

Hi James,

I like the trading simulation features in the NeoTicker program and in the Ninja Trader program. Both allow you to store and replay market days very realistically. Where simulation programs tend to fall down is in simulating orders being filled in the book. If you mostly enter and exit at the market, however, that's not a concern.


602010 said...

"OBJECTIVE(1.0) -DESCRIPTIVE(0.75) -H Y P O T H E S I S (fiftyfifty) - SUBJECTIVE gossip (0.0)"...?!

Brett Steenbarger, Ph.D. said...


An alternate take: Some researchers are trying to gather objective data from the subjective gossip on boards for traders. It's not clear at this point whether such observations are sufficient to build an accurate theory about markets, from which hypotheses could be derived and tested. Perhaps the quantity of gossip is more predictive than its content.


Brian said...

Hi Brett, Always enjoy your work. I looked at TraderDNA and was wondering if you know of any way to plot past trades on a chart, buy uploading an activity report, so one can see visually their buys/sells? TraderDNA is a much more sophisticated approach but I was looking for something more basic and visual. Thanks!!

Brett Steenbarger, Ph.D. said...


Thanks for the note. I believe some trading platforms--CQG comes to mind--allows you to visually plot your trades and even the points where you work orders.