Monday, September 15, 2014

The Challenge of Cultivating Trading Intuition

If we are to ground our investment and trading decisions in sound understanding and prediction, what, then, is the role of intuition in decision-making?

The recent post on intuition began with an interesting quote from Einstein, in which he described the rational mind as a faithful servant and the intuitive mind as a sacred gift.  Indeed, to those who achieve creative insights, it does indeed seem as though those flashes of insight come from an other-worldly source.

The problem in financial markets is that so many participants lack both faithful servants and sacred gifts.  I have met many traders who claimed an edge in markets due to their superior ideas and/or their superior gut feel for markets.  In the majority of cases, it is difficult to identify a concrete process that would generate either superior insights or intuitions.  Too often, the ideas traded are casually derived and held by a consensus of participants.  What passes for gut feel is fraught with recency bias, overconfidence, and a host of other cognitive distortions.  

If I claimed to be a great sprinter but never won a race and could not demonstrate superior running times on the track, you might think me to be delusional.  Many traders, facing years of poor results, will make comparable claims.  When pressed to identify the source of their (seemingly hidden) talent, they frequently will cite intuition and a superior market feel.  If only situational forces--psychological ones, the evils of algorithmic market manipulators--could be tamed, they maintain, their talents would finally shine through.

Suppose we encountered an island tribe in which the natives looked to the shapes of passing clouds for clues as to their destiny.  Dark clouds foretold an ominous future; a cloudless sky suggested a sunny path ahead.  We would no doubt chalk up these practices as the superstitious beliefs of a primitive culture.  Now imagine we encountered a tribe of financial participants who look to ancient numerical sequences or shapes on charts for signs of the future of asset prices.  This we chalk up to "technical analysis" and place on the program of expo events.

I have never felt a particular desire for psychedelic drugs and other mind-altering substances.  Reality itself is weird enough.

And yet there is far more to intuition than superstition and cognitive bias.  Some of the most successful traders I've known have demonstrated--year after year, over thousands of trades--a superior ability to read short-term patterns in markets.  What sets these intuitive traders apart from those who are merely deluded?  Where do they get their sacred gift?

Let's consider an analogy.  Suppose I identified a person who had a superior intuitive ability to forecast the weather.  This person could tell when it was going to rain, when temperatures would become cooler, and when a storm was approaching--all with well-above chance levels of success.  If we were to dissect the success of our forecaster, we would find out that he or she had developed a feel for factors that truly are related to weather changes:  shifts in wind velocity, shifts in air pressure, changes in humidity, etc.  In other words, the intuition is grounded in pattern recognition, and the pattern recognition is grounded in variables that are objectively related to the intuited outcomes.

Compare this with a would-be weather forecaster who based predictions upon a preordained set of wave patterns linking temperatures and precipitation.  

I've had the honor of watching several skilled intuitive traders in the process of their trading.  To a person, they focus on market factors that (perhaps unbeknownst to them) have been extensively studied and documented in the academic finance research literature:  factors such as momentum and volatility.  They are like the skilled weather forecaster:  they have developed a sensitivity to changes in the environment and the correlation of those changes with future outcomes.  Their sacred gift is the result of experiencing so many situations that pattern recognition becomes their faithful servant.

As a psychologist for over 30 years, I have many intuitive insights into the people I work with.  I do not have intuitive insights into ice skating or plasma physics.  Intuition comes from experience--but it has to be the right experience.  Years of exposure to random inputs will not bring sacred gifts.  Intuitions are only valid if pattern recognition captures variables that truly are causally related to anticipated outcomes.

If this is true, much of traditional trader education is misguided.  To build a trader's intuition, we should expose the developing trader to truly predictive variables and their co-occurrence across many market situations.  Simple price and volume charts or depth of market displays are ill-designed for this purpose.  If the variables that are most predictive are ones like momentum, correlation, and sentiment, then we need to develop displays that capture how momentum, correlation and sentiment behave under a variety of market conditions.  It's not that price charts are wholly unrelated to these things; it's that if we wanted the clearest and least ambiguous displays of the most predictive variables, we would not rely upon a price chart. 

Intuition can be a controversial topic.  On one hand you have advocates of intuition who claim a mystical source for their insights.  On the other hand, you have rationalists who deny the validity of intuition altogether.  There is a science to cultivating intuition, but I suspect it's in its infancy.  We can only develop a valid feel for things if we are systematically exposed to things of demonstrable validity.  The recent posts on identifying drivers of short-term markets is but the first step in a larger developmental effort to cultivate sacred gifts from faithful servitude. 

Further Reading: Underconfidence and Overconfidence in Trading