EA Backtesting

Trading Systems Expert Advisors and Automated Trading
1

Hi everyone,

I’ve been creating a wide variety of EAs for testing of trading strategies, and I’ve had a few hiccups in understanding the results.

Specifically - time duration and their results.

If I take a strategy, and backtest to the last 12 months - I get a loss, over the last 6mths a win, over the last 18mth a win, and over 12 years big wins, but 24 months a loss … If I optimize, one time frame may see improvements, whereas others will see failures.

My main strategies are Trend following, long term positions (daily charts) but how should I be profiling my backtest data to find good results over the long term?

I don’t want to end up over-optimizing/curve fitting just to find it fails miserably on the live account instead.

Any tips on how/what to look for as a cross reference from period to period?

Some items are obvious:
[ul]
[li]Look at Entry/Exits
[/li][li]Look at largest Wins/Losses
[/li][li]Compare one strategy at a time :slight_smile:
[/li][li]Money management is key (drawdowns, losses)
[/li][li]Looking at the timeframe which is incurring losses for specific market conditions
[/li][li]
[/li][/ul]

But is there something more I should look at so that I can ensure a greater chance of success?

2

Hi Poctim

Been there done that and don’t really know the answer.

What I used to do is find the worst periods of trading then just run my EA across those dates in visual mode and try to understand why it was failing.

It maybe the strategy and indicators you are using that cause your strategy to drift in and out of phase with the market.

Look for periods when the market goes very quiet and the bars get small, range bound or multiple Doji with almost no high or low. At times like these if you are using % of account to calculate lot size then lot size will grow exponentially and any whipsawing will devastate your account very quickly.

My latest EA also uses a ATR(14) to get the average bar size and if bar size goes below 18 pips it won’t trade. For my particular strategy backtesting showed bar sizes greater than 17 – 20 to be best.

If your EA trades long and short make sure the trades are about 50/50 and there is no major imbalance. Also click on short trades only and test to see how it performs only trading short, then try long trades only.

Also be aware that the spread size can effect results and running the same backtest ten minutes later will give different results if the spread has changed.

Normally the spread size is set to current in the strategy tester and it will backtest with whatever the spread was when you started the backtest. Also be aware that the spread size at weekends can be huge when the servers are off line. Also bank holidays and holidays in general cause slow markets that can increase the spread size. I use a maximum spread size filter on all my EA’s to protect against entering a trade during periods of excessive spread size.

Lately I have concluded that the market is so volatile, with one bar up and the next down that my latest EA has only the ATR(14) for bar size. Most indicators lag so you can’t use them.

My trade entry and exit now is down entirely to price action and use of pending orders.

Hope this helps.

Trader9.

3

Hi Poctim,

Just had a second thought. Don’t forget the market now is nothing like it was ten years ago. Before 2009 the FX market had some great trends and all that changed with the banking crisis and more automated trading now than back then.

You might want to consider just backtesting for the last year or six months and going with that and then re-backtesting each month with the new months data to re-check the parameter you are using still work going forward.

Trader9.

4

Thanks Trader 9 - Great tips, specially the ATR idea, I like that

Finding all sorts of new ways to use indicators

Also noticed how different the markets were, so I’ve ignored everything 2008 and prior - just way too different. For the most part, I back test 1 year to the day. Sometimes as far back as Jan 2013. Just seems more reliable considering what I see in the recent past.

I’ve also noticed the ranging nature of many pairs, not so much in certain time frames, but these trend techniques have HORRID entries (half way up or the top of each trend before it reverses) - seems all I did with stellar results was curve-fit some useless entries - time to try something new :slight_smile:

Any resources on Price Action you could recommend? Seems there are many people speaking of this type of trade lately - I took a look, but don’t have much info - and I don’t have time to scalp, at most a few hours a day after 8pm ET. So position trading or swing trading would be the best type of trading for me right now. I like the balance of swing trading :slight_smile:

5

Hi Poctim,

I only backtest using Jan 2013 to date as I think it is more representative of the current market conditions.

[B]Burning the candles at both ends[/B]

Look at a 4 hour chart of EURJPY and you will see all the last 24 hours of price action are sideways. My system has placed stop orders above and below this range bound pattern by 0.4 * the ATR(14) the stops are 2 * Atr away, TP is 2.8 * Atr away. However, the stop trails and as the market moves the stop orders follow so risk reward is not as bad as it looks.

This system I have called “Burning the candles at both ends”. However, I hope not to end up burning my fingers and my account.

My forex EA system is based on trading a volatile directionless market.

Having sat at the computer for many hours, it seemed to me that the current forex market is directionless in the short term and that any system based on lagging indicators is doomed to failure.

So, this system is based on price action only, with the only indicator used being the ATR which is used to establish the average bar range over the last fourteen periods.

Timeframe is four hours or could be daily as market movement in one direction for a period of time is essential for the system to work.

In the current market where consecutive bars often trade in opposite directions and random breakouts happen frequently. I have chosen to try and capture those random breakout moves and any micro trend that may follow.

In order to do this I would place a stop order above and below every new bar, based on the last bar size. In this way contracting sideways markets are avoided as the bar size will normally stay below a specific bar size and all trading will stop until once again the bar size reaches a minimum size.

This function will also form part of the money management strategy as trading in a very narrow range will lead to whipsaw losses with increased lot sizes if percentage of account trading is being used.

Placing the buy and sell stop orders too close is as bad as placing them too far away, so market volatility needs to added into the equation. So in this respect a minimum bar size is used to prevent whipsaw losses and allow room for the trade to breath.

A counter move would close any existing open market order and trade in the new direction. At the beginning of a new bar any pending stop orders are closed and a new ones opened.

Wells Wilders ATR was chosen to establish the current bar size based on current fourteen period volatility and is also used to establish placement of entry points, initial stops, break even, trailing stop and take profit parameters. In this way some measure of market volatility can be accounted for and whipsaw losses in range bound markets can hopefully be reduced to a minimum.

Initial testing on limited open only prices on H4 for 2013/14 data suggests a minimum bar size of 18 pips or greater as measured by ATR(14).

Entry size 1.2 means buy stop and sell stop to be placed 1.2 * Atr above and below last candle high and low. The same for initial stop loss, minimum profit, trailing stop and take profit, these are all multiples of the Atr at trade entry time. Drawdown (DD) values seem reasonable too.

A spread size filter of less than 4 pips was also added to stop any trading when the spread size was above this. However, for some pairs like GBPJPY and GBPAUD this was increased to 7 pips as 6 pips is normal for these two pairs.

Any constructive thoughts on the above would be welcomed.

Pair: GBPUSD
Gain/Trades: 2742/115
Min bar size: 18
Entry size: 1.2
Initial stop loss: 1.2
Minimum profit: 1
Trailing stop: 1.6
Take profit: 3.4
Max spread: 4
Draw Down: 302

Sorry there was a table of default values for about 8 pairs, but I was unable to add to this post. I am currently demo trading this on ten pairs and live trading GBPUSD.

Regards Trader9

6

Hi Poctim,

After my post giving you suggestions on how to improve your backtesting, I got to thinking about my own backtesting.

Testing version 4 of my EA I noticed it made a gain of £1876.00 on 380 trades, with a drawdown of £678.35 from a £1000.00 account using 0.5% of account on every trade using Jan 2013 to May 2014 H4 data on EURJPY.

Not bad with 189 shorts, 39% profitable and 191 long trades 48% profitable. A good balance of 50/50 trading and a profit.

However, I decided to alter my EA to only trade long or short via an external switch. Extern bool TradeLongOnly = true; and added that to the long and short trade criteria. True for long only and false for short only, using && TradeLongOnly==true or && TradeLongOnly==false.

Going long only, produced a gain of £2261 for 158 trades and a £332 drawdown. Short only, £596 gain on 34 trades with a drawdown of £212. Overall thats £2857 gain for only 192 trades and total drawdown of only 544.

Broken down thats 158 long trades 69% profitable and 34 short trades 47% profitable, quite an improvement.

All this shows what is obvious if you look at the weekly chart of EURJPY it has been going up since 2013, thus the 158 longs and only 34 shorts.

So using two EA’s one long and one short over the same data improved profitability by £981 for 188 trades less and a reduced drawdown by £134. Thats 50% more gain and less strain.

It all makes sense as any single EA trading in two directions with shared parameters is always going to be a compromise. Even if short and long had their own parameters there is bound to be conflicts as the market moves.

Hopefully diversifying in this way will also make the equity curve more robust.

Regards, Trader9

7

Is bactestint accurate test for our EA???
Because the result in real and backtest is different. I’ve EA and really good result in real trade, but in bactest is really bad…
I use this EA on liteforex, robo forex and some other brokers.

8

Hi sistemfx,

In a word NO.

Half the problem lies in using open bar data only. You should only use this for specific EA’s designed to use open bar only. You could use it to get some rough parameters for your EA, but then re-backtest using every tick to confirm the results.

If I think I have a winning system, I will then use every tick data and visual mode so I can see what is really happening.

Regards, Trader9

9

Hi Poctim/sistemfx,

As a result of suggesting to Poctim a few days ago to test his EA for both short and long trades and a balance of about 50/50 would be a good sign.

However, in the last few days I have first modified my EA to only trade in one direction at a time via an external switch i.e. TradeOnlyLong==true or false. This produced better overall results but had the disadvantage of needing two EA to trade long and short.

In the past I have always used a single EA to trade in both directions. I did this by using two sets of entry criteria and closing any open trade when a new signal indicated a trade in the opposite direction. However, I now believe this was a mistake as new testing has shown that leaving the previous trade open returns overall better results.

As an example if I were long and I get a short signal I would open the short trade and let the long trade run its course. Backtesting has convinced me that this is more profitable.

My trading methodology is to place buy stop and sell stop orders above and below the the last bar traded using the ATR(14) as a gauge to the average bar size. I aim to trade any random breakout to a bigger bar size and any mini trend that may follow. Stop loss size is linked to bar size, so a bar size below a certain value can cause lot sizes to get too big if you use percentage of account on every trade. So I also limit the minimum bar size that I would trade.

Rather that close any un-triggered pending orders on the next new bar open, I update the existing pending stop orders with the new entry price, stop loss and take profit values. However, the lot size cant be changed so if the bar size drops below my limit then the pending order is closed. Thus when the bar size once again increases above my threshold a new stop order is generated with a new lot size.

To cut a long story short by using all of the above I have once again combined long and short trading into one EA with each having its own parameters and allowing trading in both directions at once should it arise.

The results are amazing as the EA is much better at following the trend and in a rising market will make more long trades and a falling market more short trades.

TesterGraph.jpg700×71 16.9 KB

Optimisation is much improved with a very high proportion of the trades falling into the same area and the equity curve is a lot smoother than before.

EquityCurve.jpg700×71 7.6 KB

Regards, Trader9.

10

Lads,

All looks and sounds a bit complicated for my liking. Bot’s are just the ducks nuts for testing an evaluating a strategy but for the development of manual trading strategy only. If you need to optimize to get the bot working then the strategy is probably doomed. With that said, if you have sat there, watched hour upon hour upon hour of visual testing, you should get a pretty good feel to when the markets are kind and allow your bot to scrap out some pips and when you should turn it off. And that’s the key, if you wouldn’t trade the market in the here and now, turn your bot off. Remember, your the expert trader, its just an adviser. Believe in yourself first

11

I would tend to agree bobillbrowne, though I think some automation could save the need to constantly watch charts as well for some patterns or triggers - like notify me on a pin bar or engulfing bar sort of thing.

At the same time, some trades or order modifications can probably be fully automated but do not result to a LARGE return I suspect. The above strategy I tried seems to fail more times than succeed and each pair is different, so I suspect I only curve fitted the results. Change some timeframes, get different results - and the way the market is now, that system is only successful if you get that one BIG win, otherwise it’s a ton of losses - never know when that will happen, so it’s very unstable by itself.

More digging to do yet :slight_smile:

12

Bro, its called horses for courses. Would build an indicator for price movement, a script to trail an order, an ea to [B]test [/B]a strategy. But you are the trader bro, sooner or later you need to open a live account and put theory to the test. its the only way bro!

13

Hi All,

I think I have lost the thread here.

I build my EA’s to trade for me in the market while I go and do something else. I always found the good trades always seemed to happen when I was asleep.

Even if I was staring at the screen all day would I trade consistently in keeping with my rules. Would I get greedy, fearful be overcome with emotions.

No, I am outside mowing the lawn, down the pub or sound asleep while my EA’s trades live for me.

Why would you not trust a simple system that returns an average win of £20.35 and an average loss of £5.30, 53.8% of the time. The system has an overall expectancy of 1.48 over 306 trades using a £1,000.00 account. H4 USDCAD.

Surely, that is why we build our EA’s, so they can trade and we can play.

I currently have this EA trading live on a small account eight different pairs both long and short.

Why would it be any other way?

What did I build it for if not this?

Or am I missing something???

Regards, Trader9.

14

Bro not missing a thing. Bot’s as a automated trading systems are very high risk. This doesn’t mean there isn’t a place for them. I have my own personal bot working away in the background scrapping out a few pips here, a few pips there. But I am a far better trader than my bot cause I can adjust my strategy to the here and now of market conditions. My bot can’t. Best I can do is turn it off if I don’t like the current market conditions.

With that said I would of never developed my trading strategy if it wasn’t for the bots I built. As a research tool they are invaluable to us mug, rank and file trader.

So try switching off your bot and trading your system yourself, I think you will be pleasantly surprised at who is the better trader

15

I agree - finding that balance of bot and actual trading is important - like it or not, we can’t be at the screen 24/7 to trade, but a bot can get those extra few pips - but we also can’t just “fire and forget” the bot, need to monitor market conditions, adjust for things the bot was never designed to do.

Being the better trader is also necessary since you need to figure out why the bot is failing WHEN it fails. It’s guaranteed to happen after all. To code it, gotta know your system, what it’s meant for and how to use it - at the same time, I think a bot can have better entries/exists at times depending on how it’s coded.

When a human will have a delay that could cost them a few pips, bots won’t have that (if well coded to your rules) and in the end, it’s very biased as to what type of system you’re using. Trend, Range, Indicators, Price Action, News, Fundamentals, etc etc etc - all these things can make a huge difference, back testing or live - lots to learn and play with :slight_smile:

16

Hi Poctim,

I was thinking that you could use an EA to find your trades for you, but then enter manually when alerted. Except at 03:00 in the morning of course. Your Ea could handle this too.

Also an EA can keep you in a trade longer and stop you from being knocked out of a trade.

For example if you don’t trail your initial stop but check on every new bar that the last bar close was above your intended exit. This avoids being taken out by a spike that then finally closes above your stop level and you stay in the trade longer.
Lets say you entered long on close above a SMA and your initial stop was placed below the SMA, which you planned to trail the stop on.
You don’t trail the stop on the SMA you just check after every new bar to see that there is no close below the SMA. If there is you just exit at the market. This makes you spike proof once the market has moved a little way in your favour.

Whipsaw losses can be limited if you allow your EA to trade in both directions at the same time. In this way you will be long and short during a period of consolidation until the market takes off in whatever direction, then one trade will already be it the market and the other will be stopped out. Better to have one trade stopped than five or six.

Also I use the ATR to establish a minimum bar size if the bar size gets too small then I stop trading and close any stop orders. Very small bar sizes can lead to whipsaw losses and large lot sizes if you use a percentage of your account on every not a fixed lot size.

Another trick I use is to set a maximum spread size for each pair. If spread sizes get to big I stop trading as a large spread size can kill your edge and reduce your profits.

Regards, Trader9.

17

Hi All,

I have found a new way to improve my EA’s by up to 50% and make them quicker to backtest using the MT4 strategy tester.

The trades will better reflect the market you are in by going long more often if the market is going up, or down more often if the market is falling.

I basically trade the four hour charts, as they seem the most profitable for my style of trading. I place stop orders above and below the market either using multiples of ATR(14) above/below the previous high/low or use period highs & period lows.

If my stop orders are not triggered and don’t become market orders during the current four hour period, then my EA updates the stop order entry price, take profit and stop loss values to reflect the changed market conditions when the new bar opens.

The first breakthrough I have already mentioned, is I allow my trades to complete their cycle. So if my BuyStop is triggered and I am long the market and there is a market move that triggers a SellStop on the same pair, I do not close the long, but allow both trades to work out. This proves to be more profitable over the long run as the new short position could have failed as a trade and stopped my long trade from going on to bigger profits.

Ok, improvement number two. I have moved my minimum profit and trailing stop code inside the once every four hour loop. So any move to break even or of trailing my stop only happens once every four hours on the new bar open.

This gives two benefits, the first is it protects your stop if the next four hour period is very very volatile and it also means that you can get good results using the strategy tester on “Open prices only”. Because if you then test using “Every tick” using the same parameters the results are very similar and you only need to make one quick check.

Basically this change quickly puts you in the “Ball Park” of best value parameters for your EA. You could then fine tune using every tick if you wish, but don’t over optimise.

Regards, Trader9.