Tuesday, October 09, 2007

How Do You Know You Have A Trading Edge?

Once again, the comments to recent posts have been most enlightening. The topic for the present post came from a penetrating set of questions asked by NQ Trader in response to my "Wonderland" post.

NQ Trader was asking an epistemological question: a question about the state of our knowledge as traders. We often hear of trading edges, but how do we *know* we have an edge when we trade? How do we know that results aren't merely the result of chance?

It seems to me that there are two answers to that question:

1) Defining Edge in Terms of Backtesting - One tradition examines trading patterns over a historical period that includes a variety of market conditions (bull swings, bear swings, high volatility, low volatility) and determines whether the distribution of price changes following these patterns displays a positive expectancy (i.e., a non-random directional bias). Such an approach is most commonly seen in the development and testing of mechanical trading system with such software as TradeStation. The definition of edge is thus historically based. True, the future may not mirror the past, and care must be taken to not curve-fit historical tests. Still, the backtesting of patterns over market history has led to significant profits for a variety of quantitative funds.

2) Defining Edge in Terms of Trading Outcomes - Defining the edge of a discretionary trader is a somewhat trickier matter. The discretionary trader, by definition, is not relying upon fixed signals for trading decisions. Instead, he or she is reading market patterns from experience and acting accordingly. The edge of the successful discretionary trader is something akin to the edge of a highly successful athlete: it may be felt, but it is ultimately known only in retrospect. When we analyze a discretionary trader's results, we can see if the trader differs from chance in terms of the proportion of winning trades, the earning of profits, etc. For a trader who makes, say, 1000 trades, we can even conduct simulations and determine the probability that a random series of 1000 trades would achieve or exceed that trader's results. Indeed, by treating the discretionary trader as if he or she was a trading system, we can analyze results and identify an edge.

Speaking solely for myself and my own trading, I occupy a space somewhat between these two definitions of edge. I investigate historical patterns in the markets and factor those into my decision making. Ultimately, however, this factoring is discretionary and my decisions to enter and exit trades are made on a discretionary basis as unfolding market conditions dictate.

Let's take an example from the current market:

I tend to seek patterns with an edge by asking myself: "What is distinctive about the market's recent behavior?" I then test to see how the market has behaved over the past several years when that distinctive element has been present.

On Monday, for example, we made an inside day. I went back to 2004 in the S&P 500 Index (SPY) and found 119 occasions (out of 945 trading days) of inside days.

The day after the inside day, SPY averaged a loss of -.09% (51 up, 68 down). That's notably weaker than the average one-day gain of .06% (470 up, 356 down) for the remainder of the sample.

What happens, however, when the inside day follows a strong up day (as is the recent case)? It turns out that there have been 31 occasions since 2004 in which an inside day has followed a daily rise in SPY of over .50%. The next day, SPY has averaged a loss of -.26%, with only 7 occasions up and 24 down. That is quite a negative skew (which can be formally established with the use of statistical tests).

When I find patterns such as this--particularly multiple patterns pointing in the same direction--that provides a framework for thinking about the next day's trade. I then wait for the market open and see how the market is trading relative to value, how traders are hitting bids and lifting offers, etc. If I see signs of early weakness--buying that cannot, say, move the market above its overnight highs--I will act upon the historical pattern and try to profit from the edge.

Do I *know* I have an edge with such a trade? I may feel confident in my reading of the current day's trading patterns, and I may feel confident in the historical pattern I'm leaning on. Ultimately, however, the arbiter of whether or not I possess an edge lies in my trading results. What is the likelihood that those results could have been obtained randomly? That, it seems to me, is the gold standard.

If my results are consistent with those achievable by chance, then either I'm trading methods and patterns without an edge or my execution is erasing the edge contained within my methods and patterns.

In other words, we either have a logical problem (no edge to our methods) or a psychological one (inability to capitalize on an existing edge).

In the end, edge boils down to non-randomness, whether we're testing historical patterns or present market performance--and whether we're testing mechanical systems or discretionary traders.


Historical Patterns as a Heads Up in Trading


Brandon Wilhite said...

Dr. Brett,

Epistemology is very important to my trading, so I love seeing posts on it. I also really enjoyed NQTrader's comments on the topic.

I would propose that a trader's knowledge of his edge can come in three different varieties: 1) through an application of logic and mathematics to find relationships that must hold true (as in GlobeTrader's excellent example), 2) through the application of statistical studies to determine a probability and consistency of success over time, and 3) through intuition which is developed through market experience and exposure to the first 2 sources of knowledge.

Along with each type of knowledge we can make some statements about an objective level of certainty that goes with them (likely, but not necessarily, leading to more psychological ease). The first type does by and far give the trader the largest level of certainty, the obstacle here is the trader's comfort with mathematics and all of the complex details of their market. The second level gives a higher level of certainty, and in fact the certainty can be quantified objectively (I would say the obstacle here is time and tools). The third level gives the least amount of certainty, which is not to say it's an inadequate source of knowledge (Is the obstacle here time and capital?).

This is not to say that traders will experience these levels of certainty psychologically in this order. In fact, I suppose they could be turned upside down. Objectively however, I would argue that these various types of knowledge have the corresponding certainty attached to them (sophisticated philosophical arguments notwithstanding).

This is the framework which I use to deal with NQTrader's very good questions. I thought I'd offer up this classification scheme for the other categorically-minded traders out there :) . Obviously much more could be said on the subject, like the practical application of these categories.


elegy said...

This is something that has frustrated me for a long time. I am convinced that every trader should have scientific proof of their edge.

I want to do testing/backtesting but I am not a programmer. I know there are various "easy" versions of programming language, and my particular wonderful execution program (NinjaTrader) has its own proprietary programming language, and I have heard lots of pros/cons of EasyLanguage, but I am still uncertain in which language I should invest so hugely.

I also have some very experienced traders telling me that backtesting is an art/science of its own, not unlike trading itself, and it's ridiculous to think that anytime soon you can be expert enough at programming, particularly for trading. So either hire someone expensive and good (how to find them?) or forget it.

Manual backtesting seems to be an even bigger investment of time/effort, and it won't get any easier over time (unlike computerized testing).

What would you do if you were in my shoes?

Brandon Wilhite said...


I realize your question probably wasn't directed to me, but I'd like to offer a few suggestions. 1) backtesting *is* an art and science in itself and there's tons to learn in this area, there are some good books on the subject, and a lot of free material on the internet 2) don't throw out too quickly the idea of manual backtesting, at least as a starting out point, it will help you to get a good feel for your markets and learn about why certain indicators don't work very well 3) I can't speak for the NinjaTrader language, but EasyLanguage, the language used by MetaStock, and a few other of the programming languages I know of have more similarities than differences. So starting out with any one of them should get you started in the right direction. You will also benefit from learning the logic of programming a strategy, which I think is very different from a more normal type of programming.

There are in-between measures that you can also take, like using Excel and a decent database. Personally, I've used all of these options at one point or another, alone or in combination. The whole thing certainly is a learning process which can get very very involved, but that doesn't mean you have to be a master at it to get any benefit (I'd consider myself only at an intermediate stage).


Brett Steenbarger, Ph.D. said...

Thanks Brandon for the excellent insights into trading edge and backtesting. Elegy is asking great questions. I would distinguish between investigating historical patterns and developing full-blown mechanical systems. Database work is sufficient for looking into patterns, whereas the programming languages are (for the most part) needed for flexible system development.

One of my priorities on the "to do" list is to conduct a seminar on the topic of how to use Excel to identify historical patterns. More on this to come!


elegy said...

Thanks so much for answers from both of you. I'd love to attend that course, especially since I'm very comfortable with Excel. Do you do Excel backtesting using sorting and formulas, or is it more complicated? How do you get the data in Excel format?

cmescalper said...

An Edge is somthing we cant quantify or explain scientifically because as i believe an edge is a sum of a lot of thinges working together so there is not a SINGLE edge but few of them.
EDGE in cutting losing trades.
edge in defining an entry to a trade.
edge in taking profit.
edge in trader support- family, backing.

anyhow i dont think we can do backtesting on an edge because it has so many variables in it that determin it..

All the best