The Limitations of Mechanical Trading Systems

The first two posts in this series described some lessons I was learning while developing a trading system and broader reflections on theory-building in market investigation. In this last post, I'll tell you where an intensive weekend of research has left me and where I plan to go from here.

Please note that my observations are for only the markets and market data that I am observing; I cannot make with confidence judgments about other markets and data. Please also take my observations with many grains of salt, as system development is hardly my area of expertise.

What I can tell you with respect to the market variables I am investigating, which include price change, momentum, and market breadth/strength, is that a proper model for the markets would have to be one that incorporates the notion of state dependence. Over the course of years of daily data, it appears to me that the stock market exists in a finite number of states and that the predictive value of any traditional market indicator is contingent upon the market state at that time.

I suspect this is why market indicators as a whole are not good predictors of future price action, as Aronson's research suggests. What is an accurate predictive indicator in one market state--say, a narrow trading range--is not predictive in another, such as a vigorous trend. When we average performance of trading systems/indicators across a lookback period, we inevitably are averaging days from different market conditions/states. The result is that performance may look smashingly good for a period of time and then inexplicably degrade.

To make this more concrete, suppose I have a patient who has a bipolar (manic-depressive) mood disorder with episodes of psychosis (loss of contact with reality due to delusions or hallucinations). My patient has at least five different states: manic/non-psychotic; manic/psychotic; depressed/non-psychotic; depressed/psychotic; and relatively normal mood/cognition. If I try to predict how my patient will respond to something I say, I have to take into account the state that he is in. A backpat of encouragement would be read one way in a depressed state; another way in a manic one. It would have one meaning in a psychotic state; another meaning in a normal cognitive mode. Any global prediction across all states is apt to be weak. Indeed, a clinician that treated the patient the same across all conditions--a mechanical counseling system--would be hamhanded at best.

And that's the problem with mechanical trading systems with respect to the variables I'm studying. They are hamhanded. They work well when markets behave "normally" and can work surprisingly poorly when market states shift.

That isn't to say that successful systems can't be developed; I just think that such systems would have to factor in state-dependence and market context. In a sense, such a system would really embrace a set of subsystems, at least one for each distinctive market state. Their logic and validation would be non-linear: to test each subsystem, you'd have to go back to only those periods from the past that fit the state definition.

Procedurally, that means not just testing systems on the last X number of years of data. It is probably more akin to conducting nearest-neighbor modeling, where the neighbors are selected from the last X years of data based upon an objective definition of state. A set of tools for conducting such investigations can be found here; a set of common mistakes in these investigations is here.

Before using those tools, it would be helpful to develop an explanatory model that incorporates the notion of states and state dependence, per my recent post on theories and science. Such a model, if useful, would suggest candidate variables/predictors for those modeling tools. That's where I hope to be taking all of this.
.