I’m making a program that test all the combinations of a lot of pairs to see which one is the best for cointegration.
I define the “best” as the one that is more stationary (with slope==0), has more trades (like frequency of trades, a trade is counted when the pair touch the 2nd or 3rd standard deviation and returns to the mean) and has less outliers (ideally zero of them, outlier for me are the ones that exceeds the 4th standard deviation)
What I’m doing is getting all the forex pairs with small spread (less than 10 pips)
Then, I’m converting the prices like this: log(price*coefficient)
Then, I’m doing the equation and storing the result in an array
Then, I calculate the linear regression of that array
Finally, I store the data on a database
After the process is finished, I simply get the best (like defined above) result from the database, that gives me the “equation” or the hybrid fx symbol.
Then, I’m converting the prices like this: log(price*coefficient)
Hey Medisoft! It is typically either log(price) * coef or the simpler and more used price*coef. You should get very similar results from either and can compare to what you currently have. I prefer the more simple (price * coef) equation but you can check the calculations for yourself quite easily.
Just as a point of information, the slope is an indication of the amount of trend, as in a trend line. As you know, a strictly stationary series will have zero slope / trend.
A stationary time series is one whose statistical properties such as mean, variance, autocorrelation, etc. are all constant over time.
So a stable mean is one part of a stationary series.
The other important part has to do with the variance. Having a fixed variance / standard deviation is a great blessing in system design. With price time series having a fixed standard deviation is really hard to achieve due to the volatility patterns in the underlying data. But you don’t need strictly fixed std dev all the time to make a profit.
Getting back to the point made earlier regarding Old Dog’s thread, one way to test the validity of your resulting curves is to do an in-sample and out of sample test and compare the results. Your out of sample curves should not alter the slope dramatically, or the standard deviation in a major way. The most likely difficulty will be the trend / slope changing out of sample. This may indicate that the underlying symbols being used are not in fact cointegrated, possibly due to too high correlation between the pairs. Or possibly invalid assumptions in your model. There are lots of things you have to look out for.
As for your general approach it seems sound. The task in system design is a bit different than that of optimizing the stationarity of a time series, so the method of maximizing crossovers can show good results. But I suggest that really the best way to test and maximize the goodness of the curves is to create an underlying strategy, and apply the data to that strategy both in and out of sample for comparison purposes. Most stationary doesn’t necessarily equal most profitable, though more stationary usually is more profitable but also can be less volatile, which can mean less opportunity. It is a balancing act.
Note how similar sub graphs 1 and 2 are to each other. The forms are Pricecoef or log(Price)coef. The third chart log(Pricecoef) looks quite different.
By way of contrast and for the sake of completeness, I ran the same pairs with Johansen cointegration using R. Below are the first three columns of eigenvectors applied to the price charted.
R code for package urca:
m = ca.jo(p, type=c('trace'), ecdet=c('trend'), K=2, spec=c('transitory'))
Values of teststatistic and critical values of test:
test 10pct 5pct 1pct
r <= 4 | 3.18 10.49 12.25 16.26
r <= 3 | 14.15 22.76 25.32 30.45
r <= 2 | 27.05 39.06 42.44 48.45
r <= 1 | 54.03 59.14 62.99 70.05
r = 0 | [b]97.30[/b] 83.20 87.31 [b]96.58[/b]
Based on the test statistics I would conclude that only the first picture (first column of eigenvectors top of post, first row of teststatistics just above) is significant at the 99% level. This is interesting because it appears that only the 2nd picture has a stationary mean.
The 3rd picture looks sort of similar to the first two pictures in the previous post! Note, I didn’t center the charts for the final number in each column (trend.l1) which is also the intercept value in a regression.
I installed arbomat with R and the EA on a demo account.
I still don’t know how does R obtains the coefficients so fast. I suppose it solves a matrix or something like that, compared to the brute force attack I’m doing.
I provided to arbomat with some of the combinations that my own method gives to me, and I found that they shows a cointegrated chart, with zero slope, so maybe I’m doing something good.
What I need to do now is to obtain my coefficients in a faster way, because it can take hours to obtain them for only 6 forex pairs.
Arbomat is using the Engel Granger method of cointegration. That method uses regression. You can see in the code the calls using lm that show what is going on. See the onOpen method in Arbomat for the real guts of the approach.
if (allow_intercept){
Rx("model <- lm(y ~ x)"); // fit the model
}else{
Rx("model <- lm(y ~ x + 0)");
}
Note the caveats associated with using allow_intercept = true with use_diff = true discussed in the other threads. It can lead to spurious regressions. I generally favor allow_intercept=true with use_diff=false.
Have you considered using genetic optimization over the brute force approach? You provide a range of values (minimum and maximum desired parameter values) for each coef, and then randomly select from that range parameters to test. You set your tests for say 5000 runs this way and can cover fairly well the landscape to get a quicker idea of which coefs are better according to your scoring method.
Yes, I considered that, and also made my program to do that.
It is better for a big combination of symbols, like 7 or more, because brute force with 7 pairs take days to complete. With 6 or less it take less than a day to complete, and if I trade 4H it is fast enough.
By the way, I think I will need to learn more about R, to understand the code of arbomat ehehhehe.
I was thinking about using sum of vectors. Let say, I calculate the linear regression of the majors, and convert the line equation to vectors.
What I need is a resulting vector with 0 degree angle, so I can sum all the vectors, with coefficients to have a resulting vector with that zero degree angle. Maybe that will give me an equation to solve instead of brute force / genetic approach.
Don’t these test statistics show that there is no cointegration relationship at all between those pairs? r denotes the cointegration rank. Even for r <= 1, the test fails reject the null hypothesis at the 10% level.
r=0 is interpreted as there is at least one cointegrating relation, r <= 1 as there are at least two cointegrating relations etc. It’s counterintuitive, I know. See this for more:
I suggest learning the urca and vars packages as they have very good implementations of the functions you will need to perform Johansen cointegration and also run the augmented d ickey fuller (ADF) tests for validation.
I also highly suggest Bernard Pfaff’s book Analysis of Integrated and Cointegrated Time Series in R. It is more of a workbook than a textbook but it shows how to use the functions within R and essentially how to interpret the results, but not how to apply them to pairs trading. So you will need to do some supplemental learning of R (there are some time series introductions for R) and also regarding the concepts of cointegration (Carol Alexander). For the pairs trading concept you could look at something like chapter 2 from: Pairs Trading, Convergence Trading, Cointegration
I see, the r=0 line is rejecting a zero rank at the 1% level so there is significant statistical evidence that there must be one (or more) relations. The r<=1 line then tells us there is only one. I also found these notes useful (for anyone else reading): http://www.ualberta.ca/~sfossati/e509/files/class/Lec11.pdf
Out of curiosity, what was the time period you used? Hourly? And are you trading any of these relations? I’m skeptical of simple two-factor cointegration in FX but still undecided about multi-factor.
I was using 1 minute data. No I’m not trading any of them. They tend to be longer term in nature and my bent is toward shorter term strategies that can produce a smoother equity curve. When applied to smaller amounts of data cointegration tends to fall apart out of sample from my experience. My strategies are more similar to the ideas discussed in this thread based on empirical pairs trading. But I must admit I’m spending much more time in development than in trading. But the idea that you can apply coefficients to time series and create a more normalized spread line is my takeaway. If any are interested in trading using cointegration based methods I suggest taking a look at Ernie Chan’s blog: Quantitative Trading
So, what do we conclude by looking at the output? We start going down one row at a time, and compare the test statistic with the critical values. 118.26 is more than the critical values 31.2379, 33.8777,39.3693 which means that we are not able to say whether there is 0 cointegration vector. It could still be that there is more than 1 cointegrating vector. So, lets move on to the 2nd row. 114.8 is above the critical values 25.1236,27.5858,32.7172 so we cannot conclude that there is 1 cointegrating vector. But it could be possible that there are 2 or more cointegrating vectors. We move on to the 3rd row now, and get rejected again.
When the critical value is exceeded, it means that the null hypothesis cannot be rejected. In this case I understand the null hypothesis when r = 0 is rejected to mean that there are zero cointegrating relations. When the test statistic exceeds the critical value we cannot say that there are zero cointegrating relations. So we conclude that there may be 1 cointegrating relation (or more), and we continue with the next row until we get a rejection of the null hypothesis.
I wanted to reiterated a couple of points in this post that might have been overlooked by some.
I’ve been thinking about this post a lot. This is an important concept and I thank richardtannermassingill for making it. It’s sort of the inconvenient stat arb truth! But knowing it can be a means for building more stable strategies that work over the long term through time series engineering.
When A/N was not trending you could count on pretty consistent profits. Now I pay attention to A/N.
I’ve always felt that the most important thing is first creating a deterministic system - one that is predictable a priori. For instance, a sine wave is predictable. In system development having a static mean and being bounded (static standard deviation) is more than enough to make the outcome predictable. But prices don’t come that way so they must be manipulated through a bit of engineering to create a better spread upon which a simple mean reversion strategy can be applied.
Of course, some simply follow Kelton’s strategy and if they size and manage their positions properly then can do very well.
If you analyse the issue of the trending/ranging cross using the cointegration pair trading technique it’s easy to see what’s going on. Standard pair trading (Engle-Granger, linear regression, no y-intercept) fits the equation, y = beta*x,
e.g. with y = EUR-USD, x = GBP-USD, we use linear regression to fit, EU = beta*GU.
Or we can rearrange this to get, beta = EU/GU = EUR-GBP. So from the outset we expect the beta from the linear regression to be similar to EUR-GBP.
Let’s do the calculation… Here I use minute data, and do a linear regression over the past 1440 bars (~1 week). I’ve plotted EUR-GBP (blue), the value of beta from the regression (green), and for comparison the SMA of EUR-GBP over the past 1440 bars as well.
Standard cointegration pair trading says to trade when the spread, S = y - beta*x, widens. We can rewrite this equation as S/x = y/x - beta, or using currency notation, S/GU = EUR-GBP - beta.
Therefore we trade when the current EUR-GBP is a long way from the current beta. But the plot shows that beta is a proxy for the SMA of EUR-GBP. So the signals from cointegration pair trading are equivalent to just looking at a long run SMA of the cross-rate and trading when there is a deviation. When the cross trends, the strategy will break down because you’re always waiting for the cross to return to it’s long-term SMA.
This is why I don’t have great hope for pair trading in FX. Just trade a long-term SMA of the cross and be done with it. But if we jazz things up with multivariate cointegration then things could get interesting.
Very interesting analysis and presentation! I can confirm the results you show in the picture. This is essentially a visual representation of what I posted previously:
In system development having a static mean and being bounded (static standard deviation) is more than enough to make the outcome predictable. But prices don’t come that way so they must be manipulated through a bit of engineering to create a better spread upon which a simple mean reversion strategy can be applied.
In this case a better spread is one that is more stationary, bounded, and predictable. It’s funny really, that so many spend so much time developing and testing systems, when they should be spending their time with the data, and then just applying simple systems to that engineered data.
I am interested in the best way to make an indicator for an EA that can be backtested over 10 years to test this stuff. The best I can come up with is based upon an MA, where we just measure the distance of price from it and use a long period. Here are the results of the Euro with a 1000 and 6000 EMA:
The code I used in thinkorswim is given. The 6000 EMA is better since it doesn’t produce as much divergence. But since there will probably be some, it may be better to buy one then average in a certain amount of pips rather than to buy one at -6 then average in at -7 (since again the indicator may show divergence and not get to -7). When price goes back to 0 on the indicator it means it has touched the 6000 EMA. That is the normal exit of the strategy, a mean reversion.
So you would get a line for GBPUSD as well and then measure the difference between the Euro’s line and it. I’ll show this in my next post. The timeframes change the readings, so you would have to develop OB/OS levels for each timeframe you trade. Still, this way the method can be backtested.
Here the green line is the 6000 EMA and the blue line is the 1000 EMA on the upper chart. The first indicator is based on the 1000EMA and the second the 6000 EMA. EURUSD is the purple line, GBPUSD is the red line, and the yellow line is the difference between them. This yellow line is the important one, as it tells the difference between EURUSD and GBPUSD. The entry point would be at some OB/OS level, and the general exit point is when the yellow line reverts to 0.
In this second chart I just dropped the data of the two lines and only kept the difference between them. In the first indicator graph EURUSD is the 0 line, the red line is the GBPUSD difference from EURUSD, and the green line is the AUDUSD difference from EURUSD.
The second indicator graph is the difference between GBPUSD as the 0 line and AUDUSD.
The third indicator graph is the difference between USDCHF as the 0 line and USDCAD.
Lastly, has anyone thought about inversing USDCHF and trading the EURUSD USDCHF difference? This might work well. I made an indicator of the difference below. You would have made 44 pips on the trade I detailed, where both EURUSD and USDCHF would have been sold. A 31 pip loss on the latter and a 75 pip gain on the former.
It might be naive, but it seems to me since these pairs are so correlated there is a better chance to achieve the desired mean reversion. In the trade we sold EUR bought USD, sold USD bought CHF. This seems like it is simply selling EURCHF, but I tallied the trade and only came to 25 pips profit. Perhaps my calculations are off, but it doesn’t seem they would be off by that much.
