Trying to backtest and my results seem ridiculous!

Trading Systems Expert Advisors and Automated Trading
#1

But I’ve rigorously tested my code and I really think it is working as it should be, without any bugs. The original aim was to try and work out which combinations of technical analysis tools and stop-loss/take-profits produced the best results.

Disclaimer before I link to my results: I know these results are stupidly ridiculous. I’m just hoping that someone here can shed some light on this predicament: [B][U]Dropbox - STOCH-14,3,3-2011Jan-2016Mar.xlsx

Some notes on the spreadsheet:

  • Processed on EUR/USD, hourly chart from January 2011 until March 2016 inclusive

  • “Cooldown” means the minimum amount of time (hours) between trades. It is the only restriction on multiple trades - as long as the cooldown has elapsed, a trade is taken as soon as the indicator sends a signal, regardless of whether any other trades are still ongoing

  • Percentage gains (columns J to P) are all account size percentages relative to the start of the year (ie. at the start of the year, the account size will always be 100%, so if the column value is 66%, it means there was a loss of 34% that year. Conversely, if the column value is 200%, it means there was a gain of 100% that year)

My main concerns:

  • Average end-of-year account growths of 50% are already pretty ridiculous (considering this is only from the Stochastic indicator), but the top one reaches 9,000%+ growth on avereage, per year - WTF??

  • Massive stop-loss:take-profit ratios seem to vastly outperform the more conservative SL:TP ratios like 1:1 or 1:2. (The 25 pip stoploss + 200~300 take-profits appear to perform the best by far)

If you think you know what’s going on, send a comment my way, whether it be a reason why this historical backtesting bears no significance, or if these results are actually impossible and you think my code is definitely bugged, or whatever. I’m still relatively new to trading and forex and have a long way to go.

Here’s a link to my Github project which generated these results:

[B][U]github.com/Quas94/qfx-analysis[/U][/B]

The backing library used for technical analysis is TA-lib.

#2

First, well done. I love to see C++ code , and it seems well written.

I skimmed this and didn’t bother to bust out my math skills, so I my be incorrect, but your percent loss and your percent win seems off to me.

To me, it looks like you create TP and SL pairs which you send over to your strategy, but then you calculate your percent loss as your risk_percent_per_trade and i dont see you change your lot size, if its there. I believe this is incorrect because your SL is determined by your risk percent. By closing your trades based on your SL which you send in, that’s not the same percent loss as you calculate every time. Either you use your SL as a result of how many lots you trade, or you use how many lots and keep the SL constant. Your percent profit looks off as well.

EDIT : I suppose it doesn’t matter because this is just a tester so your lot availability might not matter. And it’s assume you would be trading with enough margin available.

Again I could be wrong I took a quick glance over everything and didn’t really check the math too much, I’m probably missing something.

The only other thing I might suggest is to calculate your SL and TP based if you had a high or low that hit your targets. Granted Close is fine for the most part, but you can get a little closer, for example EURUSD could easily go below your SL in 1 hour of 25 pips, but end up closing higher so your SL doesnt trigger.

EDIT 2: Another note, it doesn’t look like you compensate for FIFO. Might not be an issue with you, but some brokers don’t let you open Buy and Sell orders at the same time for the same currency. I can’t tell if you have that in the code, but it didnt look like it where your open trade logic was.

great job, keep me updated.

#3

Open a micro account with a few hundred dollars and run your program for 90 days.
If the account isn’t blown, double the initial deposit and run it for 90 more.

Rinse and repeat a few more times.
If @ the end of 9-12mo the account is still active, you’re on to something.

The proof is in the pudding.

Jake

#4

Yeah, pretty much. My SL and TP are set in stone, and I’m assuming that I have the margin required to make my position sizes large enough to cover 25 pips = 2% of account. Everything is calculated with respect to the account size at the start of the year, which is always set to 100%. So if the column under 2015 says 125%, it means there was 25% growth from the beginning of 2015 to the end of 2015.

Yup, that’s planned, but I doubt it’s the cause of the crazily inflated percentage numbers, so I haven’t worried too much about it yet.

Not too worried about this right now; because I’m really convinced that this strategy isn’t going to work. If it ended up working (huge long shot), it’d be fairly trivial to open 2 sub-accounts, 1 for longs only and 1 for shorts only.

So basically I’ve run through my code and tested it and manually tested it over and over, and I’m convinced that it’s running correctly. I made a similar thread to this one over on /r/forex on Reddit, and folks there seem to think that my code checks out, however the flaw in my project is data mining bias (which is, to my understanding, similar to “overfitting the curve” in science) - see this post in particular [B][U]reddit.com/r/Forex/comments/4hir3l/backtesting_nightmare_looking_for_insight/d2qehht[/U][/B]

Thanks for your reply, much appreciated.

This is pretty much the plan, with practise accounts. Although data mining bias probably means that it’s doomed to fail. I really just find it impossible to believe that returns like these are remotely possible, especially with such a stupidly simple strategy which is only based on the stochastic indicator.

#5

This is pretty much the plan, with practise accounts. Although data mining bias probably means that it’s doomed to fail. I really just find it impossible to believe that returns like these are remotely possible, especially with such a stupidly simple strategy which is only based on the stochastic indicator.

–You need to trade w/ a live account to truly test this initially.

#6

Did you have floating volume or fixed volume,

I think for backtesting purpose, we should use fix volume, otherwise, the result sometimes exaggerate if you do 4-5 years or more