MT4 mysterious, unexplainable backtest results?

Hey everyone,

This REALLY has me stumped. If anyone can unravel this, I’d really appreciate it!

I wrote a very simple EA that always trades with SL = TP, and never closes any of its trades. That is, it just lets them hit SL/TP. Then I backtested this EA in MT4, and the results were really bad. 65% of its trades were losses while only 35% were profitable.

This is exactly the inverse of what I’d projected. So I thought: maybe I’ve accidentally swapped my long & short orders.

Suddenly, I had a bright idea. It doesn’t matter what results I projected: since SL = TP, I can invert those bad results simply by swapping my long & short orders! Keeping the same exact entry conditions as before, the EA will now buy instead of sell, and vice versa. Every SL that is hit should instead be a TP, and vice versa.

I then backtested it over the same dates and timeframe as before. Results? 60% of its trades were losses (i.e. they hit SL), only 40% profitable (i.e. they hit TP).

Why didn’t the results invert?? Logically, I think they must! Remember, SL = TP and the entry conditions remained identical. And so I feel that MT4 is cheating me. Can anyone explain what might be going on?

'ppreciate it!
-Josh

p.s. posted this in the Newbie forum because…well…uh…I’m a newbie :slight_smile:

Hmm…I’m suspecting the spreads may be causing this :stuck_out_tongue:

The spreads are part of it, but it is also the way the pairs behave, it will be a long and complex explanation that I won’t even contemplate wasting the brain power to fashion an answer for, it’s best just to accept that that’s the way it is!

That’s a pretty good test of the structure of a price curve, and actually informs us why price curves are fractal (if you think about it a while).

Basically it shows that there’s generally a swing big enough to trigger a stop loss, more frequently than there’s a straight shot upward (or downward) to the take profit level.

That implies the market must have a fairly specific curve shape. Even more so, I’ll bet that it’s true across just about all time frames… even more closely specifying a type of curve. That solution is the fractal curve we see… or rather, the fractal curve we see is one member of the family of possible solutions.

It’s not perfect of course ~ there are news events and fundamentals and things like that… but basically the price curve is driven toward this shape, through the combined actions of all the market participants trying to exploit an edge.

Essentially every predictable, stable technical edge will eventually be hammered into nothing anyhow, by trading algorithms in a finite amount of time. Which is not to say that it’s unprofitable… rather, it’s not likely profitable forever. It’s entirely possible to craft an algorithm to seek out any and all advantages by itself, simply by giving it a wide variety of fundamental waveforms and allowing it to ‘weight’ each wave component until there is as close a match to the solution curve as you want.

Or to say that in a far simpler way: if there’s an obvious steady rising line, every trader is going to trade profitably against it until the trend stops rising. Rinse and repeat for falling trends, obvious patterns, and so on.

It is of course possible to ‘overfit’ also ~ that is, create a function that perfectly mirrors the past but is a poor predictor of the future. I’m sure some coders that create such algorithms can get into that in a bit more detail; you obviously don’t want to overfit a predictive algorithm because once you do, it starts to become more and more useless.

So where is the window for success? Certainly there are opportunities.

As best I can tell (and I am no expert!) they have to be in the following solution spaces:

a) based on market fundamentals over finite time scales (e.g. dollar trending this week, that sort of thing ~ catch it before the trend fades)

b) based on market factors, for instance, if there generally aren’t enough market participants during asian session to break through support or resistance areas established prior, that’s exploitable (of course I just made that up, but things ‘like that’ could lend an edge)

c) based on weakly arbitraged timescales (say no bank would waste its valuable time trying to scalp every last crumb of value out of the market, but those crumbs are still left on the table and worth a lot to your average scalper)

d) based on peculiarities in how the market movers (banks) trade ~ for instance the use of institutional levels, and being able to spot patterns of behaviour

e) probably other stuff like the above (again, remember I’m no expert trader!)

Ironically, the stupidest strategies such as say, buy X, wait a while, sell X… buy X, wait a while, sell X… actually can work while a market trends upwards. It might even work for months or years.

So is it the strategy… or is it knowing how good of a strategy you’ve got, and when to use it? That’s a big question right there.

you didn’t say what size SL and TP you are using just that they are equal. If they are small like 10 pips or less the spread that you need to overcome on each trade is a much larger percent of each trade. This will affect the win loss ratio a lot. The spread is going to affect long and short trades the same. Rerun the strat with a SL and TP that are over 100 pips try reversing it again I bet the results will be different. If your strat were a random entry this would make win loss closer to 50%.

Hi,

Well I couldn’t resist posting on this thread!!! Sorry chaps. But here’s the ‘low down’ (and I’ve posted about this before MANY times).

About two years ago myself and two ‘trader’ or ‘cyber’ friends (both BabyPips members) did the following experiment:

We took IDENTICAL EAs, ‘ran’ them through the MT4 Strategy Tester, using the SAME broker (demo accounts) and got different results!!! And we did this not once, not twice, but quite a few times within that particular day. Then to make matters EVEN WORSE: we did the EXACT SAME TEST using the EXACT SAME EAs and the EXACT SAME BROKER but a week later BUT using the SAME PERIOD (dates and times) as the first test. Not only did we AGAIN get different results BUT the results were DIFFERENT for each of us from the initial test!!! The ONLY difference between the three of us: me in South Africa, the other in Finland, and the other in Italy. We sent MetaQuotes the results and TO DATE have never received an answer!!!

This is ONE of the reasons why I ALWAYS ‘bang on’ about PAPER-TRADING a system. FORGET about these automated backtesting systems that will allow ‘‘optimisation’’. They’re TOTAL ‘BS’.

And FUNNY (‘odd’) that this thread was started and that I noticed it now today!!! As of RIGHT NOW (this past week): I’ve been monitoring three different demo accounts with the IDENTICAL technical trading system (I’m trying to prove a point to somebody and going into detail is not prudent here) at three different brokers. The only difference??? Their timezones. And guess what: THREE DIFFERENT SIGNALS CONTRADICTORY to each other on the daily and four-hour timeframes!!! And it’s not ‘system’ dependant either. If you’re ‘trading naked’ as ‘they’ say i.e. watching candlesticks and chart patterns: again THREE DIFFERENT CHARTS for the same instrument!!!

I say AGAIN: trade something (ANYTHING) that has a FIXED OPENING TIME AND A FIXED CLOSING TIME!!! Either that or trade spot FOREX ONLY based on the WEEKLY charts!!! The WEEKLY charts are the ONLY charts that ALL brokers’ charts will ‘agree’ on a FRIDAY at the close!!! And be careful with this too!!! There are some brokers that open trading one hour later on a Sunday than most others (what’s the bet that’s because it makes the opening gap on Sunday UNTRADEABLE at those brokers)!!!

Just my ‘fifty cents’ although to be honest: some of the replies posted here are ALSO MOST DEFINITELY ‘food for thought’!!!

Regards,

Dale.

I rest my case LOL!!

Here’s an idea:

Why don’t you check your EMAIL (PurplePatchForex)!!! LOL!!!

Regards,

Dale.

Backtesting an EA on less than H4 and up timechart is almost useless in MT4, because the error of spread and the difference between real ticks and genetic ticks is too high. Then the sample data is important and that what comes with from your broker is usually almost good for the trash can.

Not to speak about your coding capabilities (which I don’t know). Albeit, usually the issue is sitting in front of the machine …

I used to backtest EAs (a lot, not just one) over almost two years and I am still improving my sample data. Now I have something 95% reliable, but the moment of truth comes after going live for a couple of months.

A way to eliminate some of the problems mentioned above is set the EA up to only trade on bar open. This means that you can not have a normal SL and TP you need to code these into the EA in a way that they will only trigger at bar open. This also means that if your SL and TP will not be hard numbers, something like if price is lower then x.xxxx then close trade. I will leave it up to you to figure out the implications of this and coming up with your own code. This is the only way I have found to get an EAs forward test to match a back test on the same price feed data. The generated ticks are almost useless.

Hello,

Well here’s ANOTHER idea: why don’t YOU check YOUR email Mr. Gecko??? LOL!!!

Regards,

Dale.

Yeah, why did I not check that? LOL! I did now. :slight_smile:

Best regards,

BC

Wow- thanks to everyone for taking the time to respond!

Took me a second there…TOO TRUE :smiley:

That was exactly it Shr1k! And you win the bet :slight_smile:

Ahh yes, I was only operating on bar open, but forgot about the latter part there. Thanks.

Sounds like everyone agrees on this. Good to know.

Since you all were so kind/responsive (forums are a great invention, huh?), I feel obliged to say something about the concept I’m experimenting with.

One day I was looking at Heiken-Ashi charts, and the thought occurred to me that for any given bar, the probability that the very NEXT bar will have the same direction as the current one is well over 50% (typically between 70-80% for most pairs and TFs). However, for the same reason HA bars form nice trends, they are also deceiving about price action.

So I wondered what the probability is of a given bar being followed by 3 more bars of the same direction, as a function of how many bars before the current one were ALSO going that direction.

I thought I should be able to make a probability distribution of this over a recent time period, and then place trades accordingly. Will post back my results if they end up significant. Any thoughts on this? Is it a dumb idea?

~Josh

Anything that adds knowledge isn’t a dumb idea. Do find out! A ‘negative’ result is also quite useful, insofar as it allows you to focus your efforts elsewhere (moving you closer to actual solutions).

It can also be that a variety of trading strategies, just like trends, do hold true over finite timescales. In which case, the key might be to identify when they are useful. Strategies for trending markets and ranging markets don’t invalidate each other, rather, they both have periods of time when they are applicable (or not). Perhaps the key is: can we identify or predict market conditions?

I dont trust the free software that broker provides…with free commission…I know its not really free(spreads)…but sounds good!..right…free software and no commissions…mt4 is designed not to work right for the trader but always working right for the broker…buy the software and pay a commission…you won’t have these issues…that free commission and software costing you so much money you just don’t know it…trade the futures…eminis,currencies…food for thoughts