Friday, March 21, 2008

Denial as a Trading Motivation

Back in the day, I used to meet with a number of couples in counseling. Not infrequently, infidelity was a major issue bringing them to see me. In a surprising number of cases, the signs of infidelity were present long before the affair was caught. Indeed, many times there were explicit warnings from others who saw what was going on. Still, the spouse didn't want to believe the worst...made excuses...turned the other way...until evasion was no longer possible.

Denial is a powerful human motivation. We have a drive to know the world around us; less commonly appreciated is our capacity to sustain ignorance. Many times, we just don't want to know the truth, whether it's what happens in a military prison or what keeps our spouse out so late.

One particularly uncomfortable truth for traders is that their lack of profits is simply due to trading randomness. It's not a lack of discipline, a lack of trade planning, or a lack of tweaking the right indicators that create losses--all of those are relatively easy to address. No, losses are caused by trading strategies that simply do not work, and that's not so easily remedied.

It's pretty threatening to think that what you're devoting time and money to has no grounding in reality. It's much easier to simply not think about it. Like the spouse who manufactures one excuse after another in the face of a partner's erratic behavior, we construct all sorts of explanations for our shortcomings.

Or we just put the blinders on. When I started working with traders some years back, I asked them to indicate their profitability over the past year. I was shocked that most could not give me (or would not give me) a response other than a hesitant, "I'm coming close to break even." Many had simply stopped looking at their account statements. The idea of actually drilling down into their trade data and extracting information about what they were doing right and wrong was quite threatening for them.

Sadly, there are plenty of willing accomplices in this denial. Coaches eagerly assure you that "psychology" is the difference between winning and losing in markets; gurus promise you hidden market secrets that will enable you to unlock your potential; publishers crank out books on how you can succeed at trading. But no one solicits trading magazine articles, workshop presentations, or books on the topic of *lack* of edge. No one wants to hear all the first-hand stories I can tell about lives and relationships harmed and even destroyed by false hopes and promises.

That's too uncomfortable. No one can make money from it. So we don't look at it; we choose to not know. But I'm telling you truthfully: I've seen just as many lives hurt or ruined by trading as enhanced.

A while back, I was asked to submit a proposal for a workshop hosted by one of the major financial exchanges. I thought a real public service would be to present research and first-hand observations pertaining to addictive patterns among day traders: how to recognize trading addiction, and what to do about it.

The idea was shot down immediately. As it turns out, it wasn't the conference coordinator that pulled the plug, but the corporate sponsor of the event. Sponsors don't make money when people don't trade or when they trade in a more controlled fashion. People, after all, come to hear about trading as a dream, not as a nightmare.

A little while back a trader begged me to talk with him over the phone and help him with his "self-defeating" emotional patterns. In an unguarded moment, I agreed. He told me about his anger and frustration during trading and how those had led him to violate his risk management rules.

I asked the trader to give me examples of what was going on during these periods of frustration. Many of the occasions boiled down to times when he was profitable on trades, but then saw those profits reverse. I drilled down further to get examples of those trades and discovered that, even though he called himself a daytrader, many of his reversals occurred on positions held overnight. Indeed, he proudly told me his rule that limited his overnight exposure to only his most profitable daytrades.

A bit skeptical, I asked him what he based the rule on. How did he know that profitable positions intraday would become further profitable if held into the next day?

He seemed stunned that I asked. I guess we're all supposed to know that "the trend is your friend" and that you should always trade with the trend.

I quickly got on the computer, downloaded daily open-high-low-close data for the S&P 500 Index (SPY) going back a little over a year (to the start of 2007) and threw together a spreadsheet that looked at returns following up days and down days. There was no programming or advanced Excel techniques to what I did; it took all of a few minutes.

To recreate what I found: When SPY was up on the day (N = 159 trading days), the next day's open averaged a loss of -.02% (79 up, 80 down). Similarly, when SPY was up on the day, the next full trading day averaged a loss of -.13% (75 up, 84 down).

When SPY was down on the day (N = 146), the next day's open averaged a gain of .03% (86 up, 60 down). When SPY was down for the day, the next trading session as a whole averaged a gain of .11% (85 up, 61 down).

So, in other words, the trader's rule had absolutely no grounding in reality. If anything, he would have been better off fading the prior day's direction. He was becoming frustrated and angry because he was losing his profits. He was losing his profits because he was trading a setup that had no validity. Frustration wasn't causing his trading problems; his bad trading was generating (understandable) frustration.

But, for me, the eye-opener was that he had never thought to check out his rule. Even if he didn't want to crack an Excel primer and learn how to find answers for himself, he could have simply kept records of his overnight trades vs. his intraday trades and seen what was working and what wasn't.

But he didn't do that.

That's when it hit me: He didn't *want* to know.

My caller was not happy with my analysis and did not seek me out further. I didn't deliver what he wanted. He wanted a self-help psychological technique to keep him disciplined, so that his rules would make him money. He didn't want someone pointing out that his rules were invalid and that following invalid rules with discipline will simply lead to ruthless consistency in drawdowns.

As our conversation wound down, he defensively explained that he had plenty of other patterns that he traded that were valid. One of his favorites were opening gaps. Long story short, I examined the spreadsheet and told him that this, too, was coming up blank. To recapitulate, upside opening gaps led to 83 wins and 82 losses for the coming trading day; downside gaps led to 71 wins and 69 losses. There were no significant differences in the sizes of winners and losers. There was no edge there at all.

"But that's for the S&P," he said. "The gaps work for the stocks."

"Which stock would you like me to run the data on?", I asked. He said no thanks; he didn't need the data.

But the analysis wasn't the point. The point was that he was trading a belief in an edge, not an edge that he had independently validated. His entire trading strategy rested on (blind) trust in these patterns. He didn't *want* to know if the patterns were good, because that--like the spouse's actual discovery of an affair--would necessitate facing unpleasant realities and making difficult changes.

I recently started work with a trader who wrote to me in elaborate detail of recent trading losses. He immediately offered to share his account statements with me so that I could help him change what he was doing. No denial. No defensiveness. No willing blindness. Just an open kimono. I confidently predict that this trader will be successful. He's doing the hard work right now: he's facing shortcomings with eyes wide open. By owning what's worst with his trading, he'll discover the best within him.


A Cardinal Trading Virtue

The Dual Path to Trading Success


HPT said...

Excellent post Dr. Steenbarger.

Bryan said...

Great post Doc. Thank you.

Moonfrog said...

Dr. Brett, you hit right into the black dot with your writing.
As a big fan of your blog as well as your website, I often pass on the links to your sites but to my great surprise I cannot but note that not many people (traders) are very interested in the topic Psychology in combination with trading, especially not when I mention blogs where mistakes and losses are pointed out. Where I and other few people (not surprisingly many women among those) learn from your writings, most of the others react even somewhat irritated, like, please don't bother me with that stuff.
I suspect that they are afraid to lose their confidence in themselves, a confidence which is nothing more than a rollplaying scenario most of the time, as you're certainly aware of.
They are really unable to see the obvious.
Amazing. Sometimes blindness has nothing to do with eyes.

Markus said...

The right follow-up to this post might be:
Evidence-Based Technical Analysis: Applying the Scientific Method and Statistical Inference to Trading Signals by David R Aronson


Nathanael said...

I think the perspective you bring as a member of a helping profession is in some ways is at odds with the reality of speculative trading.

The structure of the industry is that spec traders will mostly fail. In the grand scheme of things, what difference does it make if some schmo is to lazy to do what is required to succeed. That should be expected. If the majority redoubles their efforts, the game would just shift and get harder, and the majority would still have no chance.
It will never change. It can’t. It sometimes seems you want to be “above” all of this by only catering your services to the few who are serious. That may be a good business model and fit your interest, however focusing on the top of the pyramid does not make the broad base disappear.

It is critical that new players be continually brought into the zero sum + cost game we call the futures markets. The promotional efforts you often seem to turn your nose up at are not some ethically shady fluke, but actually realistic and required efforts if speculative markets are to maintain liquidity and fulfill any productive purpose in society. Speculators do play a role, but It is not critical, or likely not even possible, that on net they earn a profit.

Those that fight for every last mm of advantage are a tangent and an aberration. In my opinion it is in such a person's interest to fairly promote the industry, not scare people away with doom and gloom.

Brett Steenbarger, Ph.D. said...

Thanks to all for the comments on the post.

Nathanael, I'm sorry if I gave the perception that I turn my nose up at promotional efforts to bring inexperienced traders into markets. It's not that I just turn my nose up at them. Rather, I'm actively disgusted by them, refuse to do business with them, and recognize them for the lying trash that they are.

Small traders long ago stopped being significant liquidity providers for major financial markets; there is no need to have small, untrained accounts engaging in daytrading--particularly in such leveraged arenas as spot forex and futures.

But I appreciate your point that focusing on the top of the pyramid doesn't change the abysmal failure rates at the bottom. I cannot prevent people from throwing their money away on fantasies; all I can do is refuse to participate and sound cautionary notes for those willing and able to listen.


Nathanael said...


Do you think the situation is different with the big institutions or “sophisticated” investors? institutional accounts listen to slick sales pitches as well, with buzzwords that cater to their more sophisticated pallets. The industry very much relies on salesmanship, and in a zero sum game, not everyone can win. It does not matter if they are the little guy you want to protect, or barrings bank, or the sophisticated investors in slick XYZ CTA/Hedge funds that just blew up.

I have never been to one of these promotional conferences that you are angered by, so I am missing that perspective that you have.

MidKnight said...

Post of the year I think Brett - many thanks.

With kind regards,

Monique said...

Hi Dr. Brett,

Thanks again for your insightful comments. Can you point me in the right direction where to learn how to use Excel to analyze the data such as you do?

Brett Steenbarger, Ph.D. said...

Hi Nathanael,

Yes, of course the traders at funds get the sales calls all the time. That's different than, say, banks who call on small municipalities to sell them "safe" investments that turn out to not be safe or mortgage brokers who sell funky adjustable mortgages to relatively uneducated and unsophisticated homeowners who can't hope to pay those mortgages off once the teaser rates reset.

When funds get the slick sales calls, everyone just kinda rolls their eyes. It's like doctors getting pitches from pharmaceutical reps. Whatevs. But it's when the slick sales folks promise things to relatively unsophisticated consumers that my blood boils. The forex firms that actively encourage insane leverage are among the worst in that respect. Churn and burn brokers rank up there as well.


Brett Steenbarger, Ph.D. said...

Hi Monique,

I'm afraid I don't know of texts specific to using Excel for analysis of historical market data. I taught myself with one of the "how to" Excel books at the bookstore and by experimenting with the "sort" and "filter" database functions.


Firebird said...

Dr. Steenbarger,

I've yet to meet somebody else that simply tells it like it is. Yes, it is true that many "educators" and "coaches" as well as brokers will tell you that only a minority will succeed and that most traders lose money, but this is always stated in a perverse way to lead people to believe that they will be among the chosen few (thanks, of course to their seminar, indicator, advisory service or whatever... of which no audited track record is provided indeed).

Thank you for your blog, keep up the fantastic work.


ivanhoff said...

you realize that back-testing is not 100% reliable and it could often lead to false expectations. One edge provides totally different returns in different market environment. Following a strategy that gave a profit of 100R in 2005, doesn't mean that you will achieve the same results in 2008. When one edge is overly exploited it tends to alter the market environment and its own success becomes a reason for its death.
Unfortunately the past is all we have to work with. We have to use what we know in order to understand and find out what we don't know. What software do you use for backtesting?

Seemingly Useless said...

This post resonates very strongly with my own trading experiences. I have always experienced consistent success ever since I first began trading but in the past year my strategy completely stopped working due to the implementation of the hybrid system on the NYSE. I would lose money and blame it on "market conditions" or "stupid algos" but it did not occur to me to evaluate my own trading strategy. I was ashamed of my trading records and discussing the situation. Finally after hitting one of my worst months, I came to realize by trading strategy was no longer sound and decided to completely overhaul my trading style and focus more on developing a good strategy and less on results and my losses. So far I've been making some good progress albeit slow. No one likes to fail and I believe the only remedy for denial is acceptance of "failure", which is sometimes needed for growth. I have always enjoyed your books and articles because they provide such an unique angle towards trading and the stock market.

BirdMan said...

Spot on. Keep ponding away and hopefully it will sink in and we will be better traders!
"There was no programming or advanced Excel techniques to what I did; it took all of a few minutes."
I believe this topic is on your list of things that you might do and I know some of us would find it useful...
Thanks again.

The Stock Speculator said...

I've been trading for nearly a decade. I only started to really study and analyze my equity curve 2 years ago. I was amazed at what it told me and, quite frankly, I hated the narrative. The truth was I was bleeding money and was an emotional trader.

I now monitor my P&L daily, M-O-M, YTD & Year over Year. It has helped me become very profitable (up 65% this year) and a more stable , confident trader.

You have to know how bad you are before you can get to work fixing the problems.

Great post and I just picked up your latest work TDTC. It looks like another winner!

JimRI said...

I am of the opinion that one of the major challenges in our modern world is that of people making decisions and taking action based purely on beliefs and memorized rules without questioning the their validity the consequences of following them.