By Michael Berman, Ph.D. @signal2noise.news
Before I discuss the subject of today’s post allow me to provide some background.
Since 2012 I have been allocating capital to emerging and experienced traders. The approach I have used has been the use of a proprietary scoring method which acted as a filter for choosing who to invest in, and then a robust money management programme for efficient capital allocation and risk management. I seeded more than 250 traders with more than $200 million of capital over this period.
I also founded and ran a risk and profit analysis platform which included the first robo-trading coach. We analysed more than 100,000 trading accounts in our database, so I feel like I know a little bit about assessing trading talent and identifying edge(alpha).
Allocating to Backtested Returns
Over the years I have been fed a constant stream of amazing backtest trading strategy reports. These people filled with great expectations think that they are worthy of serious dollar allocations to trade.
I have never allocated a dollar to backtested strategies where the trader cannot provide at least a year of actual trades with real money to back up the backtest. I am not saying these people are not working hard and perhaps have found something very worthwhile, sadly in the real investment world it isn’t enough for good reason.
For those of you who don’t know what a backtest is, it is a simulation where a set of rules run through price data and calculate what the returns would be if those rules were followed. You can run a complex backtest over 30 years of data in 1 second with the click of a button. No real money is involved. No fear or greed is involved. No patience or resilience is involved. No confusion or certainty is involved. No happiness or sadness is involved.
Anyone who has traded with real money knows that one goes through a host of emotions when trading real money which no backtest can factor in.
However, there is one characteristic of backtesting that almost all traders fall prey to. This is the habit of constantly optimizing the backtest to make it look good on paper with the benefit of hindsight. Let me give you an example of what they call in the industry “curve-fitting”.
Example: Curve-fitting
Let us say you are trading a strategy where you buy a stock when it is above its moving average and sell it when it is below its moving average. You run a backtest over 5 years of data with a 50-day moving average and it loses money. So, you try again this time you use a 30-day moving average, it is now making money. You try again this time with 20-days, and it is a little worse, so you think I need to find a parameter between 20 – 30 days. You do some trial and error and you come up with 26.5 days as the best moving average for the trading strategy.
The problem with the above scenario is that you have no real idea why 26.5 days was the magic number, it just worked well with the symbol in question over 5 years of data. It is quite possible that it would look totally different on multiple symbols or over a longer period than 5 years.
How much data?
I find inexperienced and sometime experienced traders fall into the trap of insufficient data, which is another form of curve-fitting. I hear people asking where they can buy or download 3 years of tick data. These people who are trading medium to long term strategies are looking in all the wrong places. There is one thing I can assure you that is characteristic of all markets. Markets are subject to regimes, periods of trending, oscillating, volatility, quiet periods, etc.
Most strategies don’t work in all regimes and its incredibly difficult, perhaps impossible to identify when a regime is changing until well after the fact.
This is why I encourage people to forget about tick data, rather spend your money or time getting longer data time series to backtest your strategies across multiple regimes. For example, if you have a trend strategy and the 3 years of data you are using is during a bull trending market then of course your backtest will look great. But you don’t know if the market is about to change from a bull trend to bear trend. That is why the more data you use the more robust your strategy will be as is more likely to have a number of regimes in the data series and you then get to see how your strategy performs in different market regimes.
Single Symbol versus a Portfolio of Symbol Backtest
There are some traders who only trade one symbol and one strategy. If this is your approach, then what I am going to say next will have limited application. I do wish to say that be very careful with trading only one symbol and strategy as there will be very lengthy periods of losing money that will put enormous strain on you emotionally and financially. I therefore recommend diversifying a strategy across a number of symbols to test for robustness. My ideal strategy will make money across a number of symbols over most periods. Let’s dive in:
I look at a single symbol strategy (50 day cross of 200 day moving average), looking only at the SP – emini future, it produces a very nice backtest. 252% total return with maxDD of 31%.
We now use the same strategy across a portfolio of 8 Symbols.
The returns are much lower but so is the MaxDD however the Sharpe Ratio is not as good as the single strategy.
I include below the performance of each symbol that is allocated an equal amount to trade at the beginning of the backtest.
Concluding comments
Market leader MetaQuotes does not provide portfolio backtesting with its software. It is probably the main culprit with what I am saying in this article. They provide a brilliant backtest engine and offer tick data accuracy, but they do not offer the ability to do a portfolio backtest. I believe this suites many of its broker clients as their customers fall prey to this behavioural bias. In the example I have shared most of you will be tempted to trade the single symbol Nasdaq future (NQ) as it performed the best over the history since 2000. This is precisely what the software and brokers want. It is much easier to find a backtest that is profitable when applying a strategy to trial and error test across many symbols. However, I would argue that you are probably better off investing across all these 8 symbols and perhaps improve some of the rules around allocating the capital. It is clear that diversification provides some smoothing of returns which could allow more leverage as it has reduced the risk profile compared to a single symbol strategy.
The above example is purely illustrative of the concept of robustness that more traders avoid due to the temptation of chasing the most favourable return on paper.
Most technical analysis software vendors today offer portfolio backtesting.