Trend follower or mean reverting: a statistical approach
Building a reliable trading system is a complex and lengthy task. Doing it the right way, aka having a framework, could support the quant trader to execute in a proper way. This is the first article of several in which I present the first analysis among others useful for building a good trading system. Python code included.
For most traders, algo strategy development is merely a means to an end — unavoidable pain to be endured on the path to making money. Trading actively is usually more exciting than developing a trading system.
Unfortunately, this leads traders to take all sorts of shortcuts and make all kinds of mistakes. While this approach makes the trader’s life easier in the short run, mistakes and shortcuts inevitably come back heavily in real time trading. And the end result is usually not good.
The good news is that most traders make similar mistakes in trading skills development, and once identified, they can be overcome and corrected. Unfortunately, eliminating these issues will not make strategy development any easier. Building a solid reliable strategy with the right approaches is much more difficult than one can think especially when shortcuts during the process are taken.
Many of us, including me, know exactly how to perform a “perfect” backtest — in theory — but when it comes to bringing the ideas to the reality, things are getting more complicated.
One of the most frequent mistakes is trying to code a strategy (perhaps the idea developed from a technical indicator or from an interesting article) tailored to the “known” market/instruments (i.e. a market that one traded a lot or if the historical data is readily available).
This is a mistake because we are trying to “force” the application of a rule that is badly suited to the particular market in lieu of market analysis and understanding its nature. We keep adding conditions and filters, until the equity line produced by the “strategy” backtest is finally pleasing to us. Many indicators, many filters and lines makes the whole strategy development (and testing) extremely complicated. Many inexperienced traders (me too!) think this is the way to develop a system. Logic is if more indicators line up and a better fit of their algorithm to the past data, the better the strategy. Nothing could be further from the truth.
The process described above leads to an over-optimization (overfitting) of the strategy, which gives the trader a false sense of confidence and for sure, it does not necessarily mean it will work with live data.
Is there a hope to be a profitable trader?
In the next sessions, I will present what I consider the essential part of analysis that every quant trader should include in his pocket: statistical market analysis.
The first step should be to try to understand the nature of the market the trader is going to operate in. A widely accepted simplification is to classify the market in Trend Following or Mean Reverting: are we in the presence of an instrument that shows continuous movements in the same direction (Trend Follower), or an instrument that after a movement returns to its long-run mean (Mean Reverting)?
The daily historical series of the VIX Index, for example, has a Mean Reverting character, while the daily historical series of Gold Future or S&P 500 have a Trend Follower character.
How do we measure if a market is trend follower or mean reverting?
Let us start with a few definitions.
Everyone in the field probably already knows what are end-of-day data (EOD) or historical data.
When we think about historical data, we are talking from the computer science perspectives about “Time Series”.
As shown in the table above, we can get a basic idea of what time series data looks like; it can be any data recorded over time in sequential order. Stock prices are an example of Time Series data as well.
Is there an analytical method to define if a market is Trend Following or Mean Reverting?
Using some techniques of time series analysis, we can investigate the market behavior. Let us observe how Mean Reverting and Trend Follower markets look.
A well known Mean Reverting market is the VIX future
The VIX future prices oscillate around an average value; there are some extreme movements but they tend to return back.
A Trend Follower market is S&P 500 Future.
As per above, we can see that Time Series and Stock Prices are strictly correlated.
Also, we have seen graphically how a mean reverting (VIX) or trend follower (S&P 500) market looks like. Now it is time to use time series analysis methods to discover the market behavior from numbers.
When we talk about a “Trend Follower” market, we are saying from the computer science perspective, that the time series is not stationary or that the statistical properties of a process generating a time series (mean, variance and covariance) do not change over time.
Augmented Dickey Fuller — ADF
There are statistical tests that we can carry out on the Time Series. These tests are able to give us useful indications on the prevalent character of time series, or if it is stationary or not. The Augmented Dickey Fuller (ADF) and the calculation of the Hurst Exponent are the two tests used most frequently.
Without diving deep into the math (Wikipedia is your friend), ADF test is fundamentally a statistical significance test: the algorithm computes a test statistic and p-value.
A “p-value” measure is used in hypothesis testing to help you support or reject the null hypothesis. The smaller the p-value, the stronger the evidence that you should reject the null hypothesis.
In a Dickey-Fuller test, the null hypothesis is the presence of unit root: unit root is a characteristic of a Time Series that makes it non-stationary.
“p-value” evaluates how well the sample data support the argument that the null hypothesis is true. Practically, we can say:
- High p-value: you can accept the null hypothesis
- Low p-value: you can reject the null hypothesis
A very low “p-value” suggests that your sample data provides enough evidence that you can reject the null hypothesis; example: a p-value less than 5% means that the probability of error that the null hypothesis is not true is less than 5% and we can reject the null hypothesis.
So, how are we going to interpret the ADF results?
Let’s assume the null hypothesis is the presence of unit root, or the time-series is not-stationary or the market is Trend Follower.
This is the final interpretation:
- p-value greater than 0.1, the data confirms the initial hypothesis: trend-follower market
- p-value lower than 0.1, the data offers evidence the market is not trend-follower
The Hurst Exponent
The Hurst exponent is used as a measure of long-term memory of a time series. It relates to the autocorrelations of the time series, and the rate at which these decrease as the lag between pairs of values increases.
Mmmm, so what?
From a Hurst Exponent perspective, a Trend Follower market is the one that increases or decreases over time (positive autocorrelation). A Mean-Reverting market is the one that fluctuates around its long-term equilibrium (negative autocorrelation).
Hurst exponent provides an answer about the main characteristic of our data.
Here are the super simplified interpretations of Hurst exponent:
- H < 0.5 | mean-reverting market.
- H = 0.5 | random walk (price movements are random and cannot be forecasted).
- H > 0.5 | trend follower market.
WRAP UP
Now we know how to numerically evaluate a market; how this information can help during strategy development?
The statistical analysis recipe includes the following steps:
- Evaluate Hurst exponent
- For stocks/market defined mean-reverting by Hurst exponent, check the ADF p-value
- Confirm mean-reverting behavior when p-value is less than 0.1
- or confirm/reject the mean-reverting behavior inspecting the plotted price data
Let‘s start with Dow Jones Industrial Average Future. Looking at the chart, in the last 5 years there is a clear up trend.
And data?
Both tests confirm the major behavior of this instrument: it is trend follower as we expected.
Now, consider a mean reverting instrument, the S&P 500 VIX Future
From the above chart, we cannot identify any strong trend (we already know VIX is mean reverting), but let’s check if the analysis confirms it.
In case of VIX, both tests confirm the initial hypothesis that VIX is a mean reverting instrument.
The same analysis can be executed not only on futures, but also on forex and it is even more interesting if applied to Stock Market.
In the notebook “medium_stocks.ipynb”, I have executed the same analysis on 486 stocks. Results can be analyzed from file “stock_results.xlsx”. The chart below is the aggregated results.
Observing the above chart, I could say:
- Stocks on long-run behave as trend follower
- On shorter time-frame, the market behavior could change but we need to consider the fact that we are observing a small amount of data
- The two tests (ADF & Hurst Exponent) presented may have discordant outcomes (gray values) and require additional inspection.
Below examples of mean-reverting stocks, based on Hurst exponent, behavior not confirmed ADF p-value; additional inspection of these charts is required.
Conclusions
In this article, I have tried to explain what I learned from many sources and what I consider a valuable tool that a quant trader should have in his or her bag. We can use presented analysis for saying how a particular instrument is, observing its data without any prior bias or prejudice and, especially for long lasting backtests, this analysis could be used as a “screener”: having more than 8000 stocks on US markets, backtesting a strategy over all stocks (brute force approach) could be really time consuming and on the some instruments a waste of time. Approaching a market in a smarter way and without any bias seems to me better approach.
In the next article I will try to confirm the discovered behavior, backtesting some simple rules: I expect that a market or a stock, identified as a trend follower, works better with strategies like “moving average crossover” or “channel breakout“ (there are many others) than a mean reverting strategies like “Bollinger bands” or RSI.
Code used is available here: https://github.com/alexgiul/medium.git
