Does everyone here do back testing for new and old strategies?
If you do or you don’t, I would love to hear your thoughts on back testing and how it has or hasn’t helped your trading.
i find it of limited (but definitely some, and important) value
it’s perfectly true that methods can backtest well and still not work in the live market, but that isn’t a reason not to backtest (as many people very wrongly allege!)
but backtesting thoroughly is still very worthwhile (for me, anyway) because you can be pretty sure that something that doesn’t do well on a backtest is going to do badly in the live market
that helps me to decide what’s worthy of further interest, attention and development and what isn’t, which is hugely timesaving
other points to note about backtesting:-
-
some methods have more reliable backtesting results than others (for example, systems using Renko or Heiken Ashi bars are notoriously unreliable, because the chart doesn’t reliably show the opening prices of the bars)
-
you can improve the reliability of backtesting by testing different historical “chunks” of data at times when the markets were behaving differently: for example, a few separate, unrelated “chunks” of a month each might be a whole lot better than a 6-month consecutive run in a strongly trending market
-
if you have a limited amount of data, it’s worth keeping an “out-of-sample” chunk of it for forward testing, as “live” as you can
-
real, live forward testing (e.g. on demo/sim) is also very valuable, in addition to backtesting
i hope this helps, Pauley - you ask good questions!
5 Likes
I’m just trying to soak up as much knowledge as possible so firing out relevant questions! Thank you for you lengthy and very helpful answers.
I was wondering if you could essentially sack off back testing and just test using live markets on a demo account. My thinking being that it could be easy for someone to subconsciously test historic charts that look like they may fit a specific strategy, therefore kinda cheating and making sure the strategy works, if you know what I mean.
3 Likes
i hear you, but i wouldn’t, myself
i find backtesting a very useful preliminary method of deciding which methods to test using live markets on a demo account - that is its main benefit to me, really
i know exactly what you mean, and i agree!
but i don’t think “abandoning backtesting altogether” is a good enough answer to that problem
4 Likes
Back-testing is useful practice in running the strategy, and in preparing in advance for handling confused set-ups, unexpected price behaviour, incomplete strategy rules etc.
Its also good for proving a strategy has a positive win rate, if that’s what it should display.
But there has to be a fundamental question here - are you seriously wishing to use a strategy that has not already been proven by somebody else’s back-testing? There are hundreds of trading strategies out there. Any strategy a private retail trader can invent, someone else already invented it.
6 Likes
Don’t worry, I’m not going to do anything stupid!
I was just curious what experienced traders did in regards to back testing.
2 Likes
around 15-20 years ago, i used a backtesting software package called ForexTester2, which came with a decade’s free data to backtest
it was outstandingly good, and super-helpful, and really great value (i think i paid just over $100 for it but that was a long time ago!)
i see from their website that it still exists now, in a new, updated version now called ForexTester5 and that there’s even a free trial available (there never used to be!)
that might well be worth a look, @Pauley1
2 Likes
I always do back tests and stress test to minimize fail risk.
2 Likes
I can’t agree with you. On the internet, you can find many “proven” crap strategy, will you put real cash into it? I think not.
there are software on the market which allow you to “generate” strategies, possibilities? Endless for retail trader.
2 Likes
Yes, there’s some rubbish out there, the internet is one place to look to find it. If you look in crap places, you will probably find crap things.
4 Likes
However, even if a strategy is proven, how does one know for sure that he is applying it properly? Backtesting for yourself, is one way.
haha
Same applies for strategies. Just because another trader can profit from it, that doesn’t guarantee I will. I should practice with it and make sure I’m using it correctly. The problem is that I may end up slowly morphing it into something else…possibly into a unprofitable system.
Just because I forward test a strategy and I don’t profit, the problem might not be the strategy. Perhaps, it’s just me.
edit: one last point. Even if I slowly morph it into an unprofitable strategy, if I keep practicing, studying, learning from my errors, I’ll turn it into a profitable strategy that matches my personality.
5 Likes
I think it depends on the strategy. If you’re looking at longer time frames and trading at close/open of candles then why wouldn’t you back test? If the strategy doesn’t work, it’ll very likely not work on the back test. Occasionally something looks great in back testing but doesn’t work going forwards, but I find they’re generally over tuned EAs.
Some strategies require timing on events such as price or an indicator crossing a threshold. If you’re doing that on the short time frames, then it’s basically impossible to back test properly other than to get an idea that the system does work. You still need to time your entries and see if you can actually stick to the system when it doesn’t go your way, that’s the hard part for me
3 Likes
Good point. For me, that’s the time to ask ¨why?´ And not in a rhetorical sense–in an analytical sense.
If it works in backtests, then try to re-trade that same set up and see what happens. What would you have to do in order to successfully trade that set up? Then try to do that again in another backtest. See what happens.
1 Like
A thing which I have noted recently has caused me some concern.
A teacher I have some respect for tends to do pretty thorough back-testing of any new strategy he is using or teaching. He varies parameters for set-up, holding period, stop-loss placement, target r:r etc. However, he has found some strategies work very well on one pair and lose consistently on another - with the same parameters. While changing some parameters makes some winners losers but some losers winners.
Running the strategy across a basket of 6 or 7 or 8 pairs produces a positive equity curve but there is no indication of which pairs to put in the basket.
The charts look identical, the specific currencies are not the problem. Completely unexpected. No explanation.
4 Likes
this is very, very weird
are they statistically significant sample-sizes, across periods of different market behaviours?
something is very wrong, somewhere
2 Likes
In my opinion, a strategy should work for a given set up, not a particular pair. However, some pairs may offer certain setups more than others. But, hey, to each his own.
Whatever strategy a trader wants to use, good for him. He just needs to understand the right time to use it, that’s all.
1 Like
Sure - he’s thorough - multiple year automated testing.
Same thing happens if he varies the time-frame. So a strategy that is profitable across a basket of pairs on D1 is also profitable across the basket on 30m, but not the same pairs are winners and not the same pairs are losers.
It contradicts accepted TA wisdom, I never expected back-testing to show this. But its a useful lesson.
2 Likes
I can only suppose that Strategy X will work on Pair Y in Year Z - but not every year.
2 Likes
no, indeed - i find it somehow “worrying reading”
if i’d found that in my own results i’d instinctively have assumed i’d made some blunder, in the backtesting process
1 Like
I used to back test a lot. Let me just say maybe that is why it took me a long time to develop as a trader. However, ever since I found comfort in the system I am currently using, I have never back tested any strategy. Maybe because I have exhausted all strategies I came across as a beginner. But the one reason why i do not back test anymore is that I have come to realize that in any trade, there are only two variables involved: Current time and Current price. That’s all I have to work with.
So back testing is kinda not my thing anymore.
Maybe sometime in the future when I want to switch or add another strategy to my trading system
2 Likes