Can anyone comment on my system's stats?

[KingKaivar]

I'm backtesting a fractal breakout system and am looking for some comments on the following stats? Is sample size too low or sufficient? Is expectancy acceptable? etc etc

Timeframe = 4H
Sample size = 400
Currency pairs analysed (100 samples each) = USDJPY, EURUSD, AUDUSD, USDCAD
Period analysed = 2001 to 2010
Expected number of trades per month = 10

I'm using a hard SL, which is 1.5*ATR(16).

With a 1.5 R:R, WinRate% is 47.38%, giving us an expectancy of 0.16R.

With a 3:1 R:R, WinRate% is 31.92%, giving us an expectancy of 0.25R.

I'm leaning towards 1.5 R:R. Even though this provides a lower expectancy, the winrate will help control drawdown and the incidence of consecutive losses.

[rhodytrader]

When you say sample size is 400, and then 100 samples each, do you mean trades? I'm not really following the math there as if that's the case then you're nowhere near 10 trades per month. Over a 10 year period you have 120 months.

Also you're using four USD pairs, so if you have overlapping trades then you are creating a multiplicative risk problem where the dollar exposure is concerned.

[KingKaivar]

Hey rhody. Yeah, samples = trades. To avoid overlapping, I tried to gather my samples from different periods. I probably should've clarified this.

For the AUDUSD, I gathererd 100 samples from 2001 to mid-2004, USDCAD mid-2004 to early 2007, EURUSD early 2007 to mid-2010, and the USDJPY 2001 to late 2004 (can't avoid overlapping at this stage as I have no data before 2001).

To calculate the expected number of trades per month, I divided 100 / duration of sampling period for each currency pair. So for the AUDUSD, 100 / (3.5 * 12 months) = 2.4 trades per month. USDCAD 100 / (2.5 * 12 months) = 3.3 trades per month. EURUSD was 2.6 trades per month, USDJPY 2.3 trades per month. This totaled up to around 10 trades per month.

[MeiHua]

i am thinking that .25 is still small. I usually look for .50 expectancy on back tested stuff. Because there will always be a degradation on unseen data and it gives more buffer room. Also this sectioning of time for different currencies is strange. Although time wise it eliminates overlapping trades in real life you will be trading those pairs at the same time and have a portfolio. If not then just pick the 1 pair you will be trading anyway. Why would you change pairs in real life every 3-4 years?? Also the most recent data is always more important. although its important to cover a lot of time the most recent years and environment is more "important" than say 10 years ago. Baseline do i think your system is ready for live trading, no. (i personally use profit factor and i am guessing that .25 expectancy is = to 1.25 profit factor, for every 1 dollar risked you receive 1.25 dollars back) 1.50 profit factor is the minimum i would look at. there is some wiggle room there, especially if you creating a portfolio with several instruments/pairs. I dont really care about times between trades but what is your flat time? How long does it take to make new equity highs? guessing at 2.5 trades a month and you could have a string of 8-10 losses. So thats like 4 months of flat time/draw down. can you psychologically deal with that?

[KingKaivar]

Why would you change pairs in real life every 3-4 years??

Hi MeiHua. Thanks for your input. I only switched pairs during the backtest to test the "robustness" of this system. This is only stemming from my own experience with backtesting. Lets say you want to gather 1000 samples, it's better to get those 1000 samples from different pairs and different time periods. This would provide better data than 1000 samples from a single pair, or multiple pairs from the same period, I would assume. I'm sure there's a technical word that describes this, but basically I wanted my samples to be as "secular" as possible, if that makes sense.

1.50 profit factor is the minimum i would look at.

Yes, I want this too. I haven't been able to design or backtest a system with this sort of return, though. But I'm still learning to optimise and hope to improve expectancy / profit factor. If it won't take too much time, can you provide or point to some resources that you use to design and optimise your trading systems?

I dont really care about times between trades but what is your flat time? How long does it take to make new equity highs?

My plan is to trade all the majors + commodities + the popular JPY cross-currencies (GBPJPY, EURJPY and AUDJPY) + gold and silver, so I don't believe I will be out for too long before a signal is generated.

[goldylox]

Post the results from the backtest.

Your WinRate% is the same as Profit Factor? or is it: Total number of wins / Total number of losses?

Profit Factor = Total Win $ / Total Loss $
eg $50000 / $30000 = PF1.66

Anyway to keep it simple run a backtest with a fixed lot of 1 lot(or 0.1) on a $10,000 account. Nice round numbers make it easy to analyze.

Once you are happy with your profit factor/settings. Then work out the optimal lot size.

Base the lot size on the Acount Drawdown.

eg if you are happy with 10% Drawdown, adjust your lot size until the backtest shows 10% drawdown.
The Profit Factor should stay the same, unless you use a money management system.

So you should have an optimal lot size for every $10,000.

[goldylox]

If you are planning on trading all pairs and metals, etc.

All pairs act differently. Test with each one independently.

Sure there are some systems that will run on all pairs. But some pairs will always perform better.

Most systems need to be tailored to a specific pair.

Try to start out as simple as you can.

[rhodytrader]

lso the most recent data is always more important. although its important to cover a lot of time the most recent years and environment is more "important" than say 10 years ago.

I'd be cautious with that view. The idea of doing a proper back test is to capture performance over a fairly broad spectrum of market environments. If you focus overmuch on the most recent period then you run the risk of your results not properly reflecting future market conditions, which inevitably will vary from recent ones at some point.

[KingKaivar]

Post the results from the backtest.

Your WinRate% is the same as Profit Factor? or is it: Total number of wins / Total number of losses?

The formulae I used...

WinRate% = total wins / total trades
Expectancy (as an expression of R) = (size of median win * winrate% - size of median loss * loserate%) / SL. Since I'm using a hard SL, median loss will always be 1R. SL = 1R.

Anyway, I'll attach a spreadsheet of my results. It's kinda raw and not intended for public consumption, so I hope it'll make sense to anyone who reads it. It's been zipped since this forum won't accept spreadsheet attachments.

The system I'm testing is very simple. Look for a fractal that forms above the 50SMA. If price doesn't retrace below the 50SMA, enter on the break of the fractal. Set your SL as 1.5*ATR(16). My SL, plus a myriad of other factors, can be optimised which is why I'm looking for feedback.

[rhodytrader]

Expectancy (as an expression of R) = (size of median win * winrate% - size of median loss * loserate%) / SL

Not sure why you're dividing by SL there. That's not part of the expectancy equation. Not that it makes a difference when SL=1, but just for the sake of correctness it shouldn't be there.