has anyone tried importing chart data from another source other than their broker to get more accurate backtests ?
I heard that dukascopy has better quality chart data but I don’t want to mess up my charts by doing this wrong, can I import a dukascopy csv file directly into mt4 and if I do this would I need to adjust for the time zone difference between dukascopy and my regular broker ?
kenny have you tried anything like that yourself ?
I read birts review it is very difficult to do, the dukascopy 1m data is in 1 hr segments.
To download a useful amount of that would be a very long drawn out process and it seems dukascopy will ban your ip if you download too many of them as they will assume you are using a downloader bot.
Apparently Alpari used to have open access to theirs but now they have moved it. It seems you need an account with them before you can access it.
I have spent all weekend trying to find a way to get reliable mt4 chart data.
I tried replacing my own chart data with the metaquotes data (I backed mine up first) because there was so many chart mismatch errors when I used mine but there is something wrong with the metaquotes own data it leaves gaping holes sometimes 2 weeks long in the weekly chart while seeming to cover those same dates perfectly fine in shorter timeframes I havent been able to figure out why this happens.
Anyway I managed to find a 126MB download of all the Alpari 1m chart data for the 7 main USD pairs in text file format for 2001 - 2009 if I can manage to import it successfully it should make for good backtesting.
What I need to know now is, what about time zone differences ?
I noticed there is a shift box in the mt4 import dialogue, which I am assuming is for shifting the time.
Alpari is in the UK so they must be using GMT but what about their summer time ?
I had been assuming I do need to offset the time the correct amount so the new chart data matches the timezone of my mt4 charts but is this really neccessary ?
I was wondering what would be the result if I just imported it as it is.
Yup, I have done it and I think it worth the effort. I have at least one EA that is spectacularly profitable with 90% testing but fails miserably with 99% testing, which is confirmed by demo trading.
The process that birt provides appears complex, but will provide a good learning experience and actually if you follow it one step at a time is pretty straight forward.
I managed to download almost 15 GB of data with no problem from Dukascopy using the scripts that he provides. Remember this is TICK level data. This was only about 2 years of data on about 10 currency pairs. This pales in comparison to the 126MB that you have from Alpari which as you pointed out is 1M data…and you cannot achieve better than 90% modelling quality with that.
If you gonna take the effort, go large.
The Dukascopy data is GMT + 0, so you must use that as your broker offset. You should also use an offline platform so you don’t get messed around by the changing market data. The best way to do this is to enable a fake proxy server(Tools>>Options>>Server Tab) and then restart the platform. This will leave the platform disconnected so you have static spreads. Obviously do this when the market is open as weekend spreads are useless.
What you say makes a lot of sense so i’m going to follow your advice I’m running birts downloader script right now the first of the files are downloading.
About the GMT offset, my mt4 is set to my brokers time zone which is GTM-6 if I set the GMT offset to zero wouldnt this make the dukascopy tick data out of sync with the charts time scale along the bottom ?
Hi SDC,
I don’t think so…my understanding is that you are dowloading and creating special “tester” history data(ie FXT file) which is different to history data that you see when you open a live chart and scroll back in time.
The tester essentially becomes totally independant of what is happening in your platform and the only time you would ever see the time scale on the bottom of a chart with this data is if you were to run a backtest in visual mode.
Well I was considering replacing my regular chart data with the dukascopy data too because my chart data has holes which is messing up my projections of fib lines and trendlines.
I was looking through the tick files I downloaded it is easy to see when the weekend starts because the bin files have zero bytes during market closes.
I noticed most of the tick files have 60 - 70 bytes and the the Friday 21:00hrs tick file usually has only 1 or 2 bytes then each file after that has zero bytes through the weekend so I assume the 21:00 file with 1 or 2 bytes is the market close which would indicate their tick data is GMT+3 which doesnt really add up becuase dukascopy is in Switzerland which is GMT+1 also their Sunday tick data starts at 22:00hrs which would indicate they are using GMT+2
Its kinda confusing.
I figured it out, the data itself is timed according to GMT like you said, the folders they are stored in are dated to GMT+3 which is a little confusing. I’m still not sure how to make the time correct for my regular account which is GMT+2 though but for backtesting its fine
Hi SDC,
The data can only be used for backtesting…think of it as a mini-broker within your broker. There is no need or nor any way to integrate it with your current broker.
Have you found any interesting results from testing EA’s yet?