Lol, that would be fantastic, wouldn’t it? Almost like winning the pools but as often as you want?
Way beyond anyone’s idea of “holy grail”? 
You sound like a very intelligent and mathematically literate person with some trading experience, so surely you must see that anything that rests on indicators and appears to give you an 84%-accurate prediction of the direction of the next bar must be very deeply flawed, somewhere along the line?
Greetings, thanks for the response. +84% prob increase in StDev in the next bar, with:
(Close[9] - MA[9]) / (Close[1] - MA[1]) = 1 or higher is the theoretical construct;
the reality depends on how much edge you are looking for in your search as you are mining the data set for a configuration of the two variables (‘features’).
Most recent test on USDCHF Buy Only -
Trained Jan 2010 to Dec 2015 to find edge configuration
Tested here in pic on USDCHF Jan 2016 to June 2017
49 signals. 73.47% win rate R:R 1:1 SL Distance = Current Close Price to SMA
Looking back 5 - 14 candles on M15 to H4 time frames simultaneously
That’s very impressive.
And not necessarily so far fetched for something data-mined.
Obviously a very impressive win-rate at 1:1, but only 49 trades, so a really small sample?
I see what you mean. But surely it’s always possible to find something that fulfils the criteria by data mining, and it doesn’t imply it’s going to work for the next 5 years just because it worked for the last 5 years? Isn’t this the exact essence of curve-fitting, in other words?
Yes you need to be careful about look back bias, but in this case the variables are pulled from 5 years data and tested for 1.5 years on a completely new data set, so a ‘training’ and ‘validation set’. There is bound to be an error in the predictive performance too but provided the edge is sufficiently high, it should balance out, especially if you spread the risk across a diverse set of uncorrelated instruments.
You should really ‘cross-validate’ with 5 years training and 1 year testing data, with a view to use your found configurations for 1 year before re calibrating continuously.
The theory generalises to all asset classes and time frames, although I have not gone as low as M1 & M5 yet.
Not to disagree here, with the underlying meaning. But, is curve fitting not also the underlying approach that many take when back testing, adjusting, and ‘improving’ a newly developed trading approach? I suppose the key question is just how far back in time, and how large does your sample have to be to realize that curve fitting is actually the reality of the market? If you curve-fitted since the start of time, so to speak, and turned a profit, then in theory you have also (unintentionally) incorporated the fundamental dynamics of the market - a sound result, do you not think?
A very laborious, time elapsed approach that I have taken since day one - a true approach to learn what drives market moves?
Finally, a thought provoking thread that steps up from the basics of learning, to quite clearly an experienced discussion. We need more of these threads, here, threads that ask real questions based on the dilemmas that real traders face 
Greetings, thank you for the post. a little more testing - will write a signal generating indicator for public use
More signals on USDCAD for Buy only on the same testing period but lower win rate.
Abs Drawdown £202 on 100K Account.
relative drawdonwn 9%
47% return at 1% risk per trade R:R 1:1, 63.59% Win Rate
Some more pairs and other asset classes to test but there is a lot of data to run through so i’m only getting through about 1 instrument in 1 to 2 days at the moment, but positive results:
USDCAD + USDCHF = 70% return so far
160 Wins 244 Trades 65.57% Weighted Win Rate
So 15.57% Edge on an R:R 1:1
The target was 20% Edge, 4.43% prediction error so far
bound to be some losers coming up
Target = +450% return per annum across multiple instruments (25 - 30)
Great job mathematical guy! I am really impressed with your mathematical terms. Everything is OK for me. But in case of behind testing, adjusting and also improving a trading approach which is developed recently isn’t the underlying approach?! How can I understand that what is the reality of the market like curve fitting or how big the sample need and also what is the good time for get back to trade?!
Thanks for the post. I my opinion you should take a common sense approach when separating your ‘training’ and ‘validation data’ if you decide to cross validate you might divide the data set into 10 groups; train with 9 and test on the one left out, doing that across the board.
A lot would also depend on the amount of data you’re combing through, the system you’re trying to build and how thorough you want to be. If I wanted to 10 fold cross validate what i’m doing here, it would probably take a few years to finally get some results and not sure how much additional value in terms of edge that would yield.
There’s lots of literature on the subject, if you are interested you might want to start here:
Are questions welcome?
In the USD/CAD example above, are you curious about why over the first 100 trades it was very steadily profitable but over the next 100 trends it made nothing at all, and in fact a small overall loss?
If you were trading it, and happened to start at trade 100, would you give up after 100 trades, assuming there was no edge at all?
Do you think 200 trades is big enough to be drawing any conclusions at all, or might a 3,000-trade history be more worthwhile?
How are you allowing for commissions and slippage, in these theoretical backtests?
Yeah sure so on USDCAD the relative drawdown is around 9% which I don’t like that much and that was due to the performance of the last 100 trades.
I’m trying to play a numbers game, so if you look at the USDCHF for example, the first 3 trades were losers, whilst on USDCAD the first 3 were winners, so if you imagine that there was some time overlap in some of those trades or you had a way to split your risk in your portfolio between them, then the net relative drawdown in that case in at the end of given time period is zero. for those two; When one pair is not doing well another pair makes up for it and overall both are hopefully profitable.
so even if i took the bad performing period on USDCAD only, with a 9% loss, USDCHF has won 23% in the same time.
The second one comes back to cross-validation, so if I am looking at 7 years data, say:
and I want to use 6 years data to test on 1 years new data, then the training set (6 yrs) Normal and the Testing Set Bold, my training and testing sequence might look like this:
if USDCAD gives around 200 trades in each test set of one year, now I have 200 x 7 = 1,400 independent test trades for USDCAD across the 7 years. Each one will have a different Win rate and Error, which will give me a set of values for variation in error that can itself be modelled…
That can come in useful for risk management, if I want to monitor Live or in a Forward Test the variation in the predictive performance of my system/model versus what should be normal, either at the granular level on each instrument, or at the macro level an overall portfolio.
In this case I am not going to 7 fold cross validate, just 1 fold. Because it’s not practical;
Each run on an M15 Candle on a single currency pair for 5 years for example, searches 149,665 candles with 200 different possible combinations of the two variables to find edge.
M15 = 149,665 candles, M30 = 74,832 candles, H1 = 37,416 candles, H4 = 9,354 candles
271,250 Candles for M15 - H4, for each of the 5,6,7,8, 9 -14 Bars looked back at = 271,250 x 10 = 2,712,500
2,712,500 x 2 Bars for each pair for Buy and Sell
The training time takes awhile. In this case it makes more sense to me to just spread the risk across 25 - 30 instruments with just 1 5-yr training and 1 year testing on each; with the hope that the multiple instruments balance each other out and have the same effect as cross validating.
Some pairs will lose money.
Are you a profitable trader, Ropunzel?
Greetings, thanks for the post.
Yes, I used to be - up and down… I don’t trade at the moment, doing research mostly and looking to launch a fund.
EURUSD Lets the team down.
22 Trades 54.55% win rate, R:R 1:1
On the upside
Greetings, for a clearer picture:
I am looking for a custom configuration on each instrument, so I have to go through the process with the 25 to 30 pairs I can find that are tradable within my spread and other parameters.
With (202 + 22 + 49) / 3 = 91 as the average run-rateam trades for the testing period. I should expect to have above 600 test trades at least. I think that should be enough before moving forward with a forward test, rather than cross validating multi-fold which will take too much time.
And then I’ll have to do the same with other asset classes, indices, commodities and others…
Should the run prove successful, then I might try cross validating for more depth on a few FX pairs first to see if it improves either the model’s predictive performance or adds some value by quantifying the risk or error variation better. If so then cross validate for all.
But one has to bear in mind that, I also have to look at the correlating between the instruments versus the correlation between the signals from the model and model that too.
And I will also have to look at the the overall risk on the portfolio when you have multiple simultaneous signals across the same or different pair; which will involve some portfolio theory to distribute the risk efficiently as well as considering the required margins to implement thone trades at any given time, especially for the capital intensive non FX products. So there is a lot look at, which would be toof long and probably boring to fit on a theory summary post like this.
A little bit bored waiting for the data to come through so decided the do something else on the D1 time frame
This time I want to predict if the D1 candle will close up with a target 85% accuracy. The target accuracy is very high so there will be few signals and with fewer D1 candles to comb through in this case I will cross-validate 6 fold.
If there is a signal at the beginning of the time frame then a trade is entered @ 1 lot with no TP or SL. at the end of the period the position is closed. So I am not taking into account position sizing or risk management; just a simple look at the win rate in this case.
Here are the results so far.
Total Trades = 77
Total Wins = 60
Weighted Win Rate = 77.92% -7.08% Prediction Error
New Test.zip (16.0 KB)
Did a quick test on EURUSD to see if generalises to the H1 Time Frame - still lots of confirmation data needed of course but it seems to have legs.
Searching for UP or Down @ Close for the next H1 Bar with 87.25% accuracy. Decided to look back 5,6,7,8,9 - 20 bars to get more signals since the accuracy requirement is high.
Trained 2010 - 2015 (Dec)
Tested 2016 - 2017 (June)
14 Trades 11 Wins 78.57% Win Rate circa -10% error
The PnL is based on the Close price, so one way it could be traded is by setting a TP & a much longer SL; to catch the High on the TP, in the event that the TP is not reached, trading just the Close will still be profitable. The SL would just be to make sure you’re not taken out by an unusual/ extra-ordinary downward spike.
Also potential use for the those crazy Binary Options traders.
EDIT: Just thinking that you could probably look at the difference between the High and the Close for each Bar, put that into a StDev of its own to be modelled. If the Auto-correlations are very good then you might be a predict where the TP should be with more accuracy, as well as position size accordingly.
This is called a “stochastic indicator”, isn’t it?
