Friday, May 19, 2006

The Most Common Trading Problem

At a gathering of investment bank trainees in suburban New York last night, I was asked my opinion of the most common problem among traders. My answer was neither fear nor greed. It was overconfidence.

Studies in behavioral finance find that about 3/4 of all traders rate their prowess as "above average", despite the obvious reality that only half of us are better than the other half. This overconfidence, moreover, affects actual trading performance. Research by Terence Odean and colleagues finds that overconfidence affects frequency of trading, which in turn contributes to poor trading results. In one study of online traders, the group of traders favored high beta (volatile) small cap companies and tended to not diversify their portfolios. Their actual trading results slightly beat the small cap index, but after trading costs were factored in, they significantly underperformed the index. The most frequent traders were the ones who underperformed the index by the greatest margin.

One of my favorite studies of overconfidence came from the London Business School. Traders were shown price patterns and asked to figure out the market's next direction and indicate their confidence in their prediction. The price patterns were generated entirely randomly. The traders with the highest confidence in their predictions traded the most frequently and incurred the greatest losses.

A completely random trader--50% right, 50% wrong--who trades once a day will have runs of five consecutive winners about six times during a year. It is difficult to not think you have the hot hand after such a string, become overconfident, and raise your trading size. Of course, the random trader will have an equal number of strings of losers. That is likely to burst confidence and lead the trader to cut trading size--assuming, of course, that he's still in the game at that point.

Is it any wonder that traders seek help from coaches and psychologists? Few of those coaches and psychologists, however, will tell the trader the truth: You're trading random patterns and your problem is overconfidence in them.


D TradeIdeas said...

Dear Brett,

Astounding insights - I'm sure your audience got a healthy dose of reality. Your argument makes more sense to me if I imagine traders facing the market with nothing other than their gut-intuition, charts, execution platform, and basic tools for trading. This includes the majority of traders that live within 1 standard deviation of the average results you mention.

My question is: can better decision support tools (used the right way) tame overconfidence and move traders towards the 'better end of the tail' within a normal distribution (i.e., towards superior returns with less of the 'id/ego')?

I agree with you that overconfidence exists in every trader - how they deal with it is largely responsible for their results. Some people posses the emotional intelligence to balance themselves, others may need decision support technology (I would argue for Trade-Ideas of course) to do it for them.

Brett Steenbarger, Ph.D. said...

Thanks for your comments, David. I think that decision support tools may or may not help overconfidence problems. If people are using better screening tools to search for random patterns, they might overtrade even more. Conversely, if they limit searches to validated strategies, the tool might help instill discipline. It's the yoking of the screening tool to empirically validated strategies and not the employment of the tools in themselves that, IMHO, is most crucial.


ANNE-MARIE said...

My goodness, if that isn't the absolute truth. My worst trading events often occur after a streak of great trades...

ivanhoff said...

Excellent piece Dr. Brett. Overtrading is a major obstacle for profitability. People tend to overtrade when they don't have a plan for the trading day/week.
If overtrading is a major issue for professional traders, lack of discipline is a major issue for developing traders.
Many people just don't have the necessary knowledge to trade profitable. Even more have the knowledge, but don't apply it on a regular basis and don't prepare for the trading day. If you are not prepared, you are most likely to engage in emotional trades with poor set ups and terrible risk/reward.
The impact of greed and fear is misunderstood by most traders. Many think that greed makes people to overstay in a certain trade and fear makes them to exit too early, just before it turns in the desired direction. On the contrary: fear makes people to exit too early from their profitable positions and greed makes people stay longer in their losing ones, as the desire to be right more often is stronger than the desire to be profitable.

By the way, "The daily trading coach" is a fantastic book and its principles can be applied in all areas of life. I truly enjoy its insights. Thank you for spending time writing it.

Manish Chauhan said...

Dr Breet

I am confused with the word "Over trading" ?

Does it mean "Trading a bigger size than you can afford" OR "Trading more often than you should" OR combination of both ?


steve said...

How common do you see "under-confidence"? Seems like that might be equally bad. Also, thanks for broadcasting your Pasadena seminar on the web!

Ryan said...

I had the opportunity to study the work by Odean in a behavioral econ seminar. Very impressive, not only for his insights, but the rigor of methodology that helped him connect specific trading problems over specific time periods with probable causes.

As usual, appreciate your insights! Hope you make it to the DC area at some point to give a seminar!

TraderSmarts said...

As usual another great post!

Dr. Brett said: “A completely random trader--50% right, 50% wrong--who trades once a day will have runs of five consecutive winners about six times during a year. It is difficult to not think you have the hot hand after such a string, become overconfident, and raise your trading size.”

I have recently been re-reading the Market Wizards series by Jack Schwager. One of the traits I have noticed among a majority of the subjects in the books is that they cut size AFTER a winning streak and increase size AFTER a losing streak. Just the opposite of what most traders do as Dr. Brett pointed out. I know from personal experience that some of my worst losses have come after a great run. I am implementing this strategy in my own trading and it would be wise for others do to the same IMO. The market is never ashamed to humble the overconfident.