A couple of members have been trading using the NEAT neural network algorithm, but I’m going down a different route and using TensorFlow which is created by google and used in many commercial things, including Tesla’s Autopilot system.
Reinforcement learning seems to be the most appropriate solution, and there are several algorithms that can be used to attain this. The principle is largely the same, we create an environment which in this case is a trading environment. The agent is given observations, in this case equity, open, high, low, close, open position price, RSI(14), SMA(20). The job of the agent is to explore the environment, make an action (buy, sell, hold) and we reward the agent based on the success of that action. The agent will try to learn how to take those actions in order to maximise its reward.
I started off with a PPO algorithm because we can run multiple environments simultaneously to make the learning more efficient, however I couldn’t get this agent to do very much at all. It always decides to do the same action regardless of input. It may well be that it has learnt enough to know what to do, so I may set it a large training task in the future.
A2C is a similar algorithm and I got similar results, so didn’t pursue this for long. DQN has been the most promising to start with. This manages to hit a sweet spot where it does very little at first, then takes a lot of trades, then starts to reduce the number of trades as it learns that it can’t always win. Catching it at the sweet spot is difficult, I’ve been saving every 200k data points, but will increase the frequency because that’s only been creating one agent that works.
On the face of it, that agent performs well. Trained from 2000 until 2010 on the Daily timeframe and then testing from 2010 until 2022 Gives over 3000 pips profit, however it made a bizarre decision to hold a trade for years at a 7500 pip loss. I think that’s because I’m differentiating between sell and close buy to try and allow the agent to realise those things are different, but if we get a sell signal with a buy position open, we should probably close it regardless.
Next steps are to try and remove those 3 big losses and ideally get more frequent trading. Then I will try it on 5 minute and hourly chart data and then different commodities.
