June 13, 2018
A note to readers: While much of this article’s content is timeless, it is from a past publication and may contain outdated information, missing links or images.
Part I
“When I was losing, they called me nuts. When I was winning they called me eccentric.”— Al McGuire, college basketball coach
“That system I bought stinks! The first three trades I made with it were all losers. I wasted a thousand bucks on that piece of junk! I’ll never trade that thing again.”
I have heard such tales over and over, whether the person is talking about a canned system they bought, newsletter recommendations or a system they developed themselves (although folks are usually less critical of things they develop themselves – more on that later).
For our system development conference call last week, we took written questions over the Internet. Unfortunately, we didn’t have time to cover all of them. One set of questions was along these lines: How do I choose between systems? How do I know if a system is broken? So to answer this line of questions, I’d like to do a series of articles on system performance. Here are some topics we’ll cover:
- What are the key criteria to use when judging a system’s performance?
- How can I choose between two competing systems?
- When does a string of losses get too long to call the system into question?
- What are acceptable drawdown levels?
- Should I look for a system with a high winning percentage or high R multiples?
- Should I buy a system or spend the time to develop my own?
To respond to those folks who throw systems out after three losses: Unless your system wins 95 percent of the time, three losses in a row is rarely something to worry about.
System Performance, Part II
This week we’ll continue with our series on trading system performance by looking into the issue of system choice. How does one choose between two competing systems?
There are two significant areas to consider when choosing between trading systems. The first is matching the system’s trading philosophy to your trading beliefs. The second area is the performance statistics of the systems. Most people spend 98 percent of their time crunching the numbers. The other two percent of system development time is spent preparing snacks. No one spends any time worrying about whether they will actually be able to trade the system they pick. So let’s spend a balanced amount of time looking at the psychology of trading the system (this weeks article) and digging through the performance numbers that will compare two systems (next week’s article).
“Yeah, I can trade that.” I’ve heard it hundreds of times, “Just show me something that works, and I’ll trade it.” We all wish it was that easy. Trading takes a combination of skill and talents. And one of the most important skills is knowing what type of systems and strategies you can trade well, day-in and day-out. So the first thing a trader must determine when choosing between systems is which one fits his or her trading style better. Here are some questions that should help you determine which system is more aligned with your trading beliefs:
- What time frame does this system use (long-term position trading? Intra-day trading? Something in between? Is this a time frame that I am very comfortable with?
- Does this system trade predominantly with the trend or take mostly counter-trend trades?
- How frequently does the system trade? Is that too much or too little for your activity level?
- How much of your investment capital will each of the systems require? Is this an amount you are comfortable with?
Be very careful if you are tempted to fall into the “I can trade it if it works” trap. Because, if the system that seems to work well on paper loses too many in a row or has one or two losses that are too big for your tastes, then you will be more than likely to toss out a good system. Understand your market beliefs and your comfort zones and you’ll be well on your way to matching them to a useful trading strategy. Next week we’ll look at how which performance measures should draw most of your attention.
System Performance, Part III
In our series on system performance, we’ll look at some of the quantitative measures you can use to compare two systems that you may be considering. In Part II we looked at matching your market beliefs with those of your system or strategy. I can’t overemphasize the importance of this aspect of your system selection! Let’s look at some of the basic quantitative measures that you need to look at when comparing systems. There are enough of these important measures to cover this week and next week.
- Winning percentage vs. High R-Multiple returns. We’ve discussed this metric before and in most instances, these are inversely proportional items (meaning that as one increases, the other decreases). The holy grail would be a system with high average R-multiples that has a very high winning percentage. While I’ve seen a few systems that do both, they invariably achieve this unusual combination by finding very infrequently occurring market conditions. These types of systems are the ones that have set-ups that come along only a few times per quarter or year. But be sure you know your ability to last through drawdowns and losing streaks. If you pick a system that generates big R-multiples, but has a winning percentage below 50, you have to have a patient demeanor. Remember to match your system with your personality and beliefs!
- Average profit per trade. This is one of my personal favorite measures. It encompasses a lot of other system characteristics including expectancy, average loss size and average win size. When combined with frequency of trade, average profit per trade can tell you more about your system than most other individual measures. While this is one of my favorites, you really need to combine it with an understanding of the next item to make sure that you don’t get fooled by one or two unusual results.
- Outsized winners. As you review trading results, especially back-tested results, keep a close eye out for really huge returns that happen only once or twice in a data run. I have seen some long-term trend following systems that put up great results because they caught a huge move in one stock or commodity. If you caught Qualcomm for a 200 point move in 1999, you could have a bunch of other average trades and still do well. What is wrong with having a big trade or two that reflect a “letting your winners run” mentality? The main thing is frequency. If these outsized winners are happening once every two or three years, it will be tough to trade your system while waiting for that next big score. Another potential problem is that the outsized gains were made when one-time events came along, like a system that was short for 9/11/2001 or when the Hunt Brothers tried to corner the silver market. The bottom line is that knowing the average returns is not enough, you have to know the individual trades that were used to generate those numbers.
Next issue we’ll look at some of the aggregate measures that are useful in comparing systems.
System Performance, Part IV
We’ve been reviewing system performance measures and last week we looked at some individual measures that are quite useful. Today we’ll look at several ways that combine or aggregate data to provide a broader measure of system performance.
- System expectancy multiplied by frequency. Van has been a great proponent of measuring a system’s expected value and he has written about it extensively. Because of the bias humans have for being right, many (if not most) people judge systems based on the percent of time the system wins without giving the ratio of the average winner versus the loser equal consideration. There are many good places to read up on expectancy including all three of Van’s books, but conceptually it measures the average expected profitability of a given system in terms of dollars won per dollar risked. To make a performance metric that is truly applicable across all instruments and timeframes, you can multiply expectancy times the frequency of the trade or investment. This will give you a “dollars per month, year, etc.” figure that you can use to compare any system. With this combination of expectancy and frequency, you can answer the question “Does that day trading system for S&P e-minis, that long-term stock trading system, or that real estate strategy that flips properties a few times a year look better?”
- Annual percent increase divided by worse case draw down. What the first measure (expectancy times frequency) won’t tell you is how much pain (or draw down) you will have to suffer to generate those average gains. A ratio I like to use is the average annual percentage gain divided by the maximum draw down. This gives us a ratio of how much we make per year divided by how much we would be down at any time during the year. Or in simple terms: How much will I have to risk losing in order to generate my average returns? Any ratio of that is less than 2:1 is suspect (do you really want to risk a 50 percent draw down to make a 50% gain?).
- Industry standard performance measures. Let’s close by looking at two composite numbers that many money managers use to measure their performance:
1. Sharpe Ratio: (system rate of return – risk-free rate of return) / standard deviation of system returns.
- The Sharp Ratio measures risk to reward by giving the returns of the system as a ratio to its standard deviation. If the system has very constant returns, it will have a high Sharpe Ratio. A system with returns that vary greatly period-to-period will have a lower Sharpe Ratio.
2. Sortino Ratio: One problem with the Sharpe Ratio is that it penalizes a system for a big up month or “good volatility”. The Sortino Ratio attempts to overcome this issue by dividing the same risk-adjusted rate of return used in the Sharpe Ratio by only the negative deviation or “bad volatility” (the downside semi-variance).
- The bottom line for measuring system performance is that you have to understand what criteria are important for your situation. And don’t just base your decision on one measure of performance. With all of the tools at our disposal for measuring performance, it is prudent to put them to use as you choose, design and use a system.