I am developing framework to create strategies with machine learning. The problem I am facing now is to refine how algorithm should decide which of two strategies is “better”.
Usually we have a strategy, automate it and then backtest. Having few strategies with few parameter sets we can freely compare them using multiple backtest data points. eg. Profit factor, average profit, Calmar ratio, expectancy, max drowdown etc.
With machine learning the engine expects to have one number which will mean “the higher the better” (in short). Then it will pick strategies with highest “scores” and work with them further. Until now I am using profit factor as single “score”, yet it does not incorporate max drowdowns or very small number of trades.
The question is - can you help me come up with single formula to calculate score of given strategy?
I may have misunderstood machine learning but that sounds like a difficult ask of the strategy itself. I mean, if you asked an aircraft flying at a consistent 10,000ft. altitude how it was doing, it would always say, “Just fine”. It could not of itself say that it was likely to hit the 100 mountains that exist over 10.000ft in height; it wouldn’t know such mountains existed or how many there were.
Would it be simpler to just run the strategy in test multiple times and developing a “ruin index”, such as the percentage of 1,000 runs which featured a draw-down of greater than 25%?
Hi, when you are use one factor, this factor describes one piece of strategy and that is not enough. Consider, to use filters instead one formula. Regards Greg
Profit is not the first thing to look for in trading.
First off, open a demo account. Then research a simple strategy that’s already on babypips or the internet, make sure its as objective as possible to minimise your decision-making. Then run it in demo until you’re consistently profitable. That’s the time to open your live account and trade.
I had that issue as well. While there is no perfect answer there is the idea that what you measure is what you optimise. So if you are only measuring profit then your strategy would end up maximising profit but it may also have a 99% drawdown or other things which you do not want. You are going to have to list what you want to see and decide if it is a sliding scale or a cutoff. So you could say that profit is a sliding scale with more being better and that drawdown is a cutoff with anything above 20% being marked as bad and anything below 20% being marked as good. Then you can sort them based on profit and exclude them based on drawdown. You may even end up with a few sliding scales and deciding to normalise them before comparison so that your sort order has a bit of both mixed in. In the end it is really up to you, just remember that if you do not take a factor into account in the scoring then the end strategy could do wild things with that factor.