Laziest trading project with Neural Networks

[wilczasty]

During a break from trading, I’ve came up with compelling idea to make my laptop figure out trading strategy for me. I want my PC to learn how to trade using neural network. In this journal I will drop some updates from the project with results as well. Long starting post follows, but still hope you will enjoy.

I’ve started from refreshing my Python vocabulary, setting up the development environment (I have not coded in Python on this laptop yet) and selecting the neural network type.
NEAT algorithm felt like good fit. It is very simple to use and to understand on high level.
Below nice youtube video, in where NEAT neural network teaches itself to play flappy bird (it was an inspiration)
https://www.youtube.com/watch?v=OGHA-elMrxI

Basically, NEAT neural network takes inputs, comes up with its own numbers and recalculate inputs into one or more outputs. For forex, inputs can be datapoints to make a decision (eg. prices, indicators, current unrealized form open trade, info if trade is opened, time of day, etc.) Output will be decision on making an action - open long, open short, close long, close short, do nothing etc.

How such neural network (NN) learn? We implement “fitness function”, so NN knows how it performs. It is nothing more than a result of current network. For trading it is simple - our fitness function is realized gain, which we want to maximize.

We will give inputs, NN will come up with random factors for calculations, produce output, we will evaluate output against our fitness function (gain). Process will be repeated 50-100 times (population of AI traders ) after that, the fittest will survive! Algorithm will select these networks who perform best, take their factors, mutate a little bit and produce new breed (generation) of 50-100 AI traders… rinse and repeat.

Sounds easy, right? It is easy, but it requires good (simple) model to be able to run on laptop within acceptable time I will be sharing more details on the current model in next post.

[ria_rose]

I’m not sure if it sounds easy for me but this seems interesting! Haha. Looking forward to your updates.

[wilczasty]

The Model

I want Bob (AI Trader) to teach himself profitable trading. The Model is everything he needs to get the job done. He needs tools like data, some environment to execute trades and feedback on his performance, so he can rapidly evolve.

Data - I am training Bob (in fact all Bobs) on H1 EURUSD data since March 2016. Doing tests on all 36000 candles was VERY slow. Thus, I’ve decided, that for each training session, I will select number of points in time (sample) and show each of them to Bob separately. This is to avoid testing on very biased data (eg. we don’t want to train on ranging market only). At first tries I’ve used only SMA and price data, but now I’m showing Bob only indicator values for given candle and expecting him to draw his conclusions (BollingerBands Oscillator, CCI, MACD, ROC, RSI, Momentum = 6 inputs)

Environment - At first runs I’ve tried to allow Bob to make four decisions: open long, close long, open short, close short. Having four outputs was too slow for prototyping, so I’ve cut this down to one decision - open long. If action is triggered, Bob will open long trade on next candle, without TP, SL for 0.01 lot with fixed 2pip spread. Trade will close automatically after 5 candles and calculate the realized pips.

Feedback - This is “fitness function” mentioned in first post. I cannot select Best of Bobs based on result from single trade. He may get lucky. So each Bob trades multiple times (on the sample) and sum of all results in pips is his “fitness” score. Bobs with most pips will create new generations of better Bobs.

Does it work? Definitely you can see better Bobs evolving. With sample size 500 (500 random points between March 2016 and today) >100 population (number of Bobs in one generation) and 500 generations, I’ve managed to get average fitness from -500 pips to >300 pips. So worst Bob on average lost hundreds pips after looking at 500 different candles. Then, Bobs evolved 50000 times, and best Bob could make from the same sample > 300 pips profit.

Is this ultimate success? No. Best Bob going through whole 36000 candles took severe losses. It is probably because sample was too small (over-optimization) and some network parameters need to be adjusted. Eg. how sure Bob needs to be to trigger the trade open or how much he mutates and tries different things.

Below screen of how current Best of Bobs trade:

You can see, he developed a sense of catching bottoms and tops. First three green arrows show how well he can sometimes predict the bottom. But, there are also these painful red boxes of loss where he tried to catch a bottom, but failed. Upper green arrows shows second type of trades Bob like - buying before top. Unfortunately, I cannot see what is inside the black box mind of Bob, but he behaves like he somehow predict, that down move will end and then jumps in to catch bottom. That’s how he get burned when market push more, but that is also how he picks the bottoms. On the other hand, if upward move establishes, he sense, that it will go slightly higher and jumps in just to cash before reaching local top, which is exciting.

If you have tried similar approaches, have any questions or just want to add comment - feel free.

[Mondeoman]

Fascinating subject matter. Please continue to report / comment / advise.

[wilczasty]

This night last generation of Bobs became extinct.
So far I’ve mentioned about populations (number of Bobs) and generations (number of populations).
Within population there may be few species of Bob. After few generations, Best Bob can mutate in different ways and this is how new species are born. Population becomes divided in few subgroups which share some similar characteristics.

If given species stagnates for few generations (does not mutate or has exactly the same results) - it becomes extinct. This night last species of Bob stagnated for too long and became extinct.

What happened?

• I’ve raised the confidence level required to open the trade. So Bob opened less trades in sample and at some point could not figure out how to open trades more efficiently.
• I missed the parameter which prevents total extinction. For next tests, I will set some degree of elitism, which will ensure, that at least n Bobs will survive to next generations waiting for new species to emerge.
• I’ve allowed 15 generations of stagnation before species are extinct. This is a lot - 1 generation takes around 5-10 minutes to compute. I will reduce it to 5.

Bob - New Hope
Below screen after first generation. There are 100 species with two Bobs each. You can see only 40 on screen, but there were more below Fitness column shows what was pips result for given Bob species. Most are heavy losing, but there are few with positive number and few Bobs decided, that it’s best to not trade at all (fitness = 0).
Bobs with positive results will be considered as priority for evolution (Best of Bobs got 103 pips gain)
In the first line you can see that average fitness (gain) is -676 pips with standard deviation of 406, which means, that most likely Bobs in current generation will be losing. We should see this average and deviation being more positive in next generations as a sign of real genetic evolution.

It does not mean, we will solve the market, but we will have a neural network which is better at trading that it was before. What is great, it is learning by itself, we don’t have to show correct trades (supervised training) nor explain what indicators do, we “only” need to tell it, what does it mean to have good result in trading.

[wilczasty]

300 generations and few sample sets later Bob gets much better results (for samples)

Best profit went from 103 to 213 and as expected the average profit and it’s deviation are much better. Still, after running on whole dataset current Best of Bobs failes miserably:

In ranging market it is wonderful how Bob takes the bottoms based only on indicator readings from single point in time. Yet all profits are consumed by losses from jumping in to early and repeatedly.

Similarly in downtrend losing trades are taken.

Nice results in uptrend.

Observations / Comments / Next Steps:
I have a hypothesis, that having only points in time as samples is not proper training method. Indicators can give the same readings in strong uptrend or downtrend and Bob will treat these identically. This would explain why results are great in uptrend, mixed in range and disastrous in downtrend. As one of potential solutions would be to give difference between 200 EMA and some other one to have rough indicator on which side of market we are.

I may need to rework how the whole trading looks like. Right now Bob opens new trades as long as he see good indicator values. All trades are closed after 5 candles. Few potential changes here.

• Lengthening the closeout period from 5 candles to 10-15. If Bob catches bottoms too early maybe he needs to wait few more candles to close at least with smaller loss. This is easiest to test.
• Allow Bob to decide on trade closure himself. This means another input (he will need to know unrealized pips to make a decision), another output and risk, that Bob will stop trading after first profit in fear from loss. It happened with one of first Bobs “I’ve had my 96 pips profit in 2016 and I will trade no more!”.
• Add short positions. I wonder, if these bad trades could somehow offset each other. Adding shorts mean one or two more outputs, so it may slow down a bit.

[wilczasty]

After “small” changes new results came in the morning.
What changed?

• Trades are closed after 10 candles (changed from 5)
• Bob now can open trades both long and/or short (yay!)
• I’ve increased the sample size from 200-500 to 5000
• Increase of sample and adding short positions lengthened the computation time significantly. It forced me to reduce Bobs population to 50 and number of generations in one run to 50 as well (comparing to previous 100-200 population and 100-500 generations) Processing one generation now took anywhere between 10-30 minutes.

Sample increase proven to be crucial to deliver results. Despite having smaller population and number of generations (ie. less training and mutations) Bobs managed to get positive results on 5000 sample quite quickly. After getting the results in the morning, I’ve ran the Best of Bobs just to see how it will fail on full dataset…

And he did not fail

It is first time, when Bob achieved long term positive result!
He became much more cautious. During 2016-Today he took only 86 trades.
His strategy changed - instead of catching the tops/bottoms, now he jumps into momentum and try to milk it. Trades are opened after huge candles.

Entry points are not bad at all (even the losing ones). I would dare to say, it could be a feasible strategy to follow Bobs advice but applying some manual SL/TP/Breakeven. This network definitely deserve some extra computation time for training.

I am thinking about implementing trade closure for Bob. This will blow the training time to the roof, but it may be interesting to see results. Other thing are the stop loss and take profit, but to run these I would probably need to buy some CPU power from Google Cloud or put ice around my laptop and let it run at full capacity for few days

Bottomline is, that Bob after having relatively little training in AI Gym got better than most of living traders out there. Not pefrect, but somewhat profitable

[wilczasty]

Digging up the thread with update.
Since beginning of April, I’ve rewritten the whole code and changed the logic (…over 20 times).
Right now Bob trades as follows:

• As input, Bob take 10 day RSI, angle of 5 day regression line, percentage change in price between last bar and current closing bar.
• As output Bob can either decide to open Buy trade (buy only at this stage) or Close, if trade is open. Both open and close i performed with Open Price on next candle.

Different approaches ended with mediocre results, until I’ve come up with idea to scale the inputs to the same order of magnitude. This way, network somehow better evolved. It must have some correlation with ease of weights mutation. Now Bob has the face of Forex Monster…

Results

Bob trained on EURUSD D1 data downloaded from some website (2007-today):

Results looked too good to be true. Maybe data used was somehow faulty. I’ve downloaded EURUSD D1 data from my broker and ran using trained network. Unfortunately my broker gives me only data from Jul 2012. Results as below:

When I see such good results, I feel uneasy. It can’t be right, right? So I’ve checked this network on something absolutely different - GBPUSD H1 data:

Before coding live trading capabilities, I need to verify this on few other instruments.
If you have any ideas how to prove, that this cannot work - please let me know

[wilczasty]

Using the same data and inputs, I’ve created separate neural network to handle short trades. It seems to be much faster to evolve one network to identify long opportunities and close trade and other one to do the same with shorts.

In around 2 hours short version of Bob was rocking the charts. I’ve combined results from both networks on training data. Short results were overall much better than long ones - it may be caused by down moves being somewhat more dramatic and have more momentum. Bob measures slope, percentage change and RSI - he may rely on the momentum (or change in momentum).

Results of Long+Short on training data

Next steps:

• continue trying to invalidate these results by testing on different datasets
• prepare the code to use both networks at the same time, so it will fit to live/demo trading
• write the code to connect with my brokers API

[wilczasty]

At last I can say, that results of these crazy networks have been invalidated.
Network could see future prices and this was behind all these great results After taking away crystal ball - results were still green, but not enough to be relevant (2kpips in ~17 years)

Back to the drawing board

[igillman]

Using the OHLC of the previous 5 candles I tried to get a neural network to predict the closing price for the following candle. The best it could come up with was to give me the price of the close on the last candle of the 5. It appears as if the greatest influence in the closing price of a candle is the closing price of the previous candle.

[ponponwei]

@wilczasty

Trades are opened after huge candles.

HMM very interesting strategy!

@wilczasty

This will blow the training time to the roof

When you say this, how long are we talking about in terms of time?

[wilczasty]

@igillman

Using the OHLC of the previous 5 candles I tried to get a neural network to predict the closing price for the following candle.

That’s not the approach I’m taking here. I let the network open and close trades and it is evolving based on it’s results over some period. So I give inputs, network applies weights, biases, additional levels of manipulation and comes up if trade should be opened or not or closed or not. Instead of asking “what would be the correct move?” I’m allowing it to play the game and assess it’s own results.

@ponponwei

When you say this, how long are we talking about in terms of time?

Generation processing time depends on amount of data (number of candles), population size (how many networks are ran within single generation), number of inputs/outputs. I try to manipulate the above to have single generation processed within 60-80 seconds to get results quite fast. Running that on laptop, so probably having high end PC or bought some cloud computing power would significantly lower the times. I’m not investing into this as this is just fun and prototyping I like to have at least 100 generations ran, preferably 500. With 1 minute run its anywhere between 1h-10h to get some results. Before that, I am running very small populations to have a taste of how network may evolve (few sec generations to have very fast and inaccurate estimations) Previous unacceptable for me strategies you’re referring to, were around 20 minutes per generation, if I recall correctly.

[ponponwei]

@wilczasty

With 1 minute run its anywhere between 1h-10h to get some results.

Wow that range is huge! One to ten hours!

[Mondeoman]

@wilczasty

I like to have at least 100 generations ran,

Hi,
Is your strategy based on the Alpha Go 2017 improvement on Deep Blue of 1997?

Artificial intelligence: Google's AlphaGo beats Go master Lee Se-dol

[wilczasty]

Hi!
I love to play Go and watched live all AlphaGo matches, but I’m not sharp enough to implement the engine myself

I’m using NEAT algorithm in python.

[Mondeoman]

@wilczasty

I’m using NEAT algorithm in python

Thanks for the info. I watched the first 10 minutes of this YT vid, and bookmarked it in my “library” in OneNote.

The NEAT Algorithm is Neat

Now I understand level 2 of 10 of what you are trying to achieve.

Dumb question. I have understood you are trying to optimize an algorithm for determining a profitable outcome for all markets in all timeframes. Have you considered restricting the boundary conditions first with historical markets that you know were in strong trend first, then gradually broaden the boundary conditions to determine when the algorithm does not succeed?

[wilczasty]

Hi Mondeoman!
I’ve dropped a link to youtube in first post, which nicely show the idea of NEAT within 10 minutes with flappy bird game as example (without getting too deep into coding)

What I’m trying to do:

• Check, if network written by myself can actually learn
• Find a network (set of inputs), which will be generally profitable in the long term
• Test network on different currency pairs / timeframes to see, if it’s not overoptimized
• Reject networks which have extreme drawdowns, small profits, too low number of trades
• Check correlation of results for different networks and try to build a portfolio of networks to work in different market conditions

So generally finding a network which historically gives positive results is surprisingly easy, but it’s hard to force it to be more “tradeable” in real life. I’m not overly attach to idea, that strategy should perform well on all markets and timeframes, but if it would work only in one pair and one timeframe - I would get suspicious.
Today new network results came out. I’ts GBPUSD H1. Since 2014 only 16 trades and it was flat between 2014-2018. So as standalone strategy it is not usable, but it may be nice addition to bucket of strategies.

[jessoprules]

following with interest

[Mondeoman]

@wilczasty

Hi Mondeoman!

High,
I like your work. It is “original thinking”. A couple of years ago, I decided to do something similar with horse racing, before I was going to apply it to Forex. I am an engineer by background and do not shy away from mathematics. But I have never been able to apply maths usefully to statistical analysis. So I took about 1,000 historical horse racing results that the “tipster” had bothered to list as his historical “bets” over a five year period, and checked about 1 in 10 of them against Racing Post historical database and they were correct. Since his service had been running for 5 years, and he published on a daily basis, I could not see how he could have faked his reported results. I even went back six months and validated his wins and losses against those he had reported.

I ran a monte carlo analysis on his results (only using an Excel pre-programmed sheet by somebody totally unrelated to the tipster. And I had read, front to back, three years prior a very colourful book called “How To Find A Black Cat in a Coal Cellar” - the truth about sports tipsters. I saw this as a validation or not of the usefulness of statistical analysis applied to the Forex market.

My father had died the year before. He used to have a twice weekly punt on “the Gee Gees” (horse racing). His success was extraordinary, to my mother, my brother and myself. A very long time ago, I recall him calling me on the phone to tell me “your old man has hit the jackpot”. He specialized in what is now called Accas - precisely termed as a Yankee. On 3p each way Yankee bets he had chosen five out of six winners - I think the multiple was about 3,000 to 1. It paid off a car loan at the time, I remember. I once left him £1,000 cash when I returned overseas asking him to put one fiftieth of it on the same punts he did for himself. After three months we talked on the phone. He said once he started to think about my £1,000 “bank” compared with his own mindset of “don’t spend more than £5 per week”, he said his “luck” had changed drastically, he had lost £100 out of the £1,000 I had left with him, so he decided to buy me Premium Bonds with the remaining £900. I have owned them ever since and never collected a win.

The result of my statistical work after a couple of weeks caused me to decide to subscribe to his tipping service. The average odds on his winners was 12:1 with a win rate of one in 6.5 averaged from the past three years. I had validated that in my Excel calculations. He averaged about one tip per day. I decided to stake £20 per bet, with a £2,000 bank. Starting with a very small loss, the bank grew by 25% very quickly. I was tracking the actual win rate versus the three year historical and noticed that the one in 6.5 was dropping and noted when it went through 1 in 7, 1 in 8 to 1 in 10. I had planned to run the account with up to a 50% maximum drawdown. I got to 45% drawdown when his win rate was down to about 1 in 11, and just before I cancelled my subscription, the tipster announced that due to losses, he was suspending his service.

I learned later that a root cause was proposed by another subscriber on a news post. I may not describe this well here, but he suspected that the change was due to Covid. Since the Covid related event calculations, there were no onsite bookies. I followed the logic. With no onsite bookies, there was a change in actual statistical outcomes that had not been a factor in most of the tipsters five year history. Whatever was this tipsters “edge”, it disappeared with the onsite bookie phenomenon - something that he previously seemed pretty good at evaporated, and had an awful impact on his outcomes.

I did learn a lot from that exercise about actual versus planned. It helped me greatly when deciding how to split up my Crypto portfolio into sections with different assumptions, in particular about utility value of NFTs or liquidity pairing.

I look forward to your future progress with the networks.