Hey guys what’s up. I want to ask something about back-testing and how it reflects on actual trading. For example, if you’d have a consistent back-tested system that gives x-amount of pips per month, what percentage of error should I count, or does it even have a error margin?
Sample:
5% error (counting spreads, off quotes, trade execution lag etc)
To come up with an answer to that you’d need to run a proper statistical analysis. No one can just give you an answer. It will depend on things like the number of observations, variance or results, etc.
5% seems a little low to me. That might cover the things you mentioned but doesn’t account for the inaccuracy in Strategy Tester and the quality of the data used for backtesting. Strategy Tester never uses tick data despite what the option says. Instead it has an algorithm to interpolate 1M data which you can read about on the MetaTrader web site. Of course if the 1M data is corrupt, that is another matter to consider. I have seen missing data and suspicious spikes in historical data which weren’t in the live data. Hope that helps.
I can just second what the others said. All depends on the system. The error of a system based on daily chart is likely less than on a M5 chart, etc. etc. Plus no back test gives you live results.
I’ll give you some numbers:
My expectation of backtest drawdown, the most important number imho, is multiplied by 2 for live control. Means, if I have for instance a max. dd of 15% in backtest I let the system run until 30% dd before I’d switch it off. Leverage decreased accordingly.
I have no expectation whatsoever in profits. Sure, I backtest and tweak the system a little for effective trading, but I am looking rather for a good sharp ratio and low dd than big profits. Also looking for a overall p/l ratio of higher than 2. Plus some other stuff which is related to my risk tolerance. Just to name a few numbers. If I would come up with the whole stuff I could probably write a book, lol.
Where now - don’t get carried away with your back testing results. They look very promising, but there still is a big difference between back test and demo and live results. The back test should be considered a means of rejecting a poor strategy, not a prediction of future results. The next step is demo trading to see how well the strategy performs in close to real world environment. Keep in mind the same kind of errors you mentioned in the first post will probably not show up in demo, but you will eliminate data quality and back testing errors.
One thing I forgot to mention in my other post - market conditions will throw off your back test results. Remember the saying - past results are no guarantee of future success. In many cases, you can’t use the past to predict the future.
CodeMeister, thanks for your input. Just to clear something up, when I say back-testing, I don’t mean Robot back-testing with Meta Trader. It’s Paper-Trading. I enter Entry, S/L and T/P figures on Excel Spreadsheet based on certain time period and systematical price action analysis. Systems are good, but the technical performance is bugging me a bit. Questions like: Will I be able to enter right on time? - Will the spread be a problem on exit and pending-order entry? - How will the order execution lag affect the overall performance? -You get the point. So what I want to do is, come with a simple statistical solution as an Error Margin Percentage of x% to subtract from the overall Pip Performance for System_X Y Z.
I hope I didn’t confuse you. Thank you again.
P.s. How can I calculate the expected value based on these figures?
Thanks for clarifying that point. Just so you know, Strategy Tester can be used to do manual testing as well as robots.
If you are doing paper testing then you margin of error is much higher than using Strategy Tester simply because your eyes will fool you. On a dead chart, you get no notion of the price movement. What you see on an entry bar could very well have taken out the trade within a few minutes instead of moving in your favour. That is just one example. The biggest problem I have with paper trading is that my eyes are attracted to entry and exit points that work and not to ones that fail. You may not have that problem, but others have mentioned it as well.
The other problem with any kind of back testing and more so with manual back testing is curve fitting. That means basing the rules and exceptions to the rules on the data you see to optimize things. Then in the real world, the data isn’t quite the same and the results suffer accordingly.
So there is no formula to use to predict anything. Once again I will repeat that back testing can be used as a means to reject a poor strategy, but it really can’t be relied upon to predict success.
CodeMeister, thank you for your experience sharing. What you’re saying is very true, you tend to favor the deals that worked while ignoring the ones that didn’t. -That’s why I wanna get some input from other people to see whether I’m right on track or not.
Hypothetically, looking at my results, even a 50% error leaves you in profit. What do you think about this?
One more thing. In my forex trading experience, I blew up my account once. During this period I really learned a lot of stuff. Things like how to deal when emotions rush in, dealing with lots of pressure etc. But one thing is for sure that, trading on Live account and Demo account is not the same. Truth to be told, only when I switched from Demo to Live, did my learning curve change in a positive direction. And, the thing is that, I can’t trade on Demo anymore. It just doesn’t stimulate my brain, it doesn’t push me. So, I’m considering to open a $500 sub-account just to live test these systems. Normally, I trade on bigger time frames( >H1). But, I never traded any mechanical system. So I’m willing to give it a try in a few weeks. Just to see what happens.
Thank you for your opinions guys. If you have any ideas or things you wanna add or ask, I’ll be around.
Off topic: I learned a lot form this guy Salman Khan. I don’t know if I’m allowed to post any links but you should definitely check his non profit academy. Khan Academy (or just google Khanacademy). Math, Statistics, Science, Finances, Physics, Currency, Banking, Economy etc. … Just thought it’d be nice to share this for those who don’t know.