I don’t think 20 is adequate at all. But apparently that is what Mark Douglas suggests in his book, [I]Trading in the Zone[/I].
I generally prefer at least 1,000 sample trades before I’m confident with a system. If there is less than 1,000, then there has to be a good reason (very high profit factor beyond the margin of error, low population etc).
Thoughts? Is Mark Douglas correct? A sample size of 20 seems awfully low…
In my view it all comes down to the frequency of how often you trade…
Should you be trading once per year, twenty years of data testing would be more than adequate. What you need to consider is that although markets never change in the way they actually operate, they do have their own behavioral cycles, and it is these cycles that need testing. For example, and to keep it simple, lets say you have a buy only system, and you have tested it over twenty trades which happened to all be in a bull market phase - I would assume the results to be rather good. However, on the flip side, you have not tested the same system on a sell market phase.
From a personal level, I test at least three years of data per interested currency pair - I get a yield of around 600 trades per year, which after filtering out reduces to about 120 feasible trades which would be taken in live circumstance.
In short, more is always better when testing, and if your doing this manually it’s the most time consuming aspect you will ever encounter when trading. It took me just over 18 months to test three years of data manually.
Good input from jezzode. I’ve read trading in the zone a few times and I think you’ve taken the 20 sample size out of context. As with any literature forex related, you should use it hypothetically and don’t follow it to the last dot. As for sample sizes, I weight them, similar to an exponential moving average vs a simple moving average. Ie the results of the current year are weighted more than last year and so on. Otherwise results from testing 5 years ago which may have been amazing vs ones from last year which were pretty average would skew the overall picture.
I wish to have an answer from you today as of 2018, this question was asked 5 years ago and knowing that the internet is full of opinions, my interests would be in an outcome of a real time experience, have you tested 20 samples only and did this work well for you?
Wow. It was just by coincidence that I was Googling and found this thread, and your post.
20 is grossly inadequate. More is better. I look for 1000+ if possible.
Broadly speaking, I’d say 20 is about right if you are a manual trader. Robo-traders may have different standards - I have no experience with that style of trading.
At the end of the day, forward-testing is the only testing that matters. Back-testing serves as a basic ‘sanity check’ of an idea or concept in current market conditions. In other words, it is intended to answer the question ‘is this strategy worth forward-testing?’.
Once that question has been answered, further back-testing has deminishing returns. In part this is because the market is constantly shifting, and the further back in time you go the less relevant the back-testing results become.
But it is also because back-testing almost always involves short-cuts; you have to assume an average spread, for example, and you probably just look at the most relevant candles rather than breaking down each candle into individual ticks. And, of course, it is very easy to look at a chart and assume that the price action would have been as obvious then as it is in hindsight.
For these reasons, I feel that back-testing for more than a bare minimum of recent price movements is largely a waste of time.
What is the point of back-testing or demo trading? Is it to prove a strategy will work? NO.
The aim is to make the strategy fail. Winning test or demo trades teach you nothing. Only when the strategy fails will you be able to understand why it failed. The next step is obvious - improve it so it doesn’t fail again this way.
But then go back to testing. Test until it fails again. In fact, make it fail. Keep on until it does fail. Then make it better again. Then go back to testing. Etc. etc. etc.