You make some good points, and a lot of what you say is true. The problem is you’ve probably spent 2 years looking at the wrong thing (Thats not intended as a critisism, I was a programmer, and Ive spent the last 10 years or so trading full time, and working on strategy development, and I suspect that I went down many of the same dead ends as you)
Mechanical trading with TA is extremely difficult. As you point out, you can take a simple TA based system (lets say, Buy when 7,3,3 stochs on the 15 min chart < 20, and close when 7,3,3 stochs > 80). You can back or forward test, and maybe even make a profit. Change the parameters to 8,3,3 and suddenly you make a loss, change paramete to 9,3,3 and you made a gain. Its just random, which perhaps isnt surprising really. You can even change the complete basis of the signal e.g. buy when stochs > 80 and find periods in which you’d be profitable
The more indicators you add as filters, the worse the problem gets as you add in even more data mining bias.
I think there are only 2 ways to go with this. You can accept that TA based approaches have a massive varience in returns, and then try to get around that by diversifying. So using the simple example above, our 7,3,3 and 9,3,3 systems where profitable, and the 8,3,3 system lost. The question then becomes does TA provide any kind of an edge, and I’d argue that it does. I’ll use an extreme example. Lets assume that you create 3 systems. The first uses a monthly chart to determine trend direction (using whatever indicator or method that you want to use), and then opens a trade at random based in that direction, closing the trade 2 hours later. The second system, uses a 1 minute chart to determine trend direction. The third system uses a 30 minute chart to determine trend direction. We could if we wished trade 10 instances of each system so we get a better average result.
I’m not suggesting that any of the above systems would be profitable (other than due to random chance), but over a larger sample size, I’d be amazed if one system wouldnt do significantly better than the other two. Simply trading with the appropriate trend would at the very least reduce your losses. Even such basic TA’s giving you a very slight edge, and with a bit of diversification and MM you can capitalise on whatever random chance throws up. Trade in the direction of positive swap, add in a few basic rules limiting losses, grab broker rebates etc and you’d probably make money. Maybe not as much as you’d want to, but probably more than most make.
The alternative way of exploiting TA is to accept that very simple mechanical rules dont work (if it was that simple anyone with a copy of trade station could make money). However, an oversold oscillator, or a stochastic cross really isnt the same thing as the same oscillator showing divergence at an area of previous support. If you combine that event with a trend reentry signal from a higher timeframe at a lower high, we are not even in the same ball park. The problem here though is these kind of set ups cant really be implimented with simple conditional IF THEN ELSE type statements, things are a bit more fuzzy, and they take experience to spot in real time.
But even then, its ultimately about a lot more than TA. Its probably more about accepting trading with TA is just playing with probabilites, and accepting whatever outcome happens. As time goes by, you get better at exploting the opportunities as they come along. Understanding when you’ve been lucky, and capitalising on it is an important skill, but its only developed through experiece.
Another argument that I’ve seen put forward is that ultimately, profitable trading is about price action, by which I mean general wave structure to define trend, price patterns, S&R etc and individual bar patterns which define entry triggers. These things are more difficult to spot in faster timeframes as there’s more noise, but indicators help make those things clearer. The guys who think they are trading with indicators arnt really, they are subconsiously trading price action. I probably agree with that to some extent.
The whole thing really is about setting realistic expectations, and expecting to find a simple indicator (or a combination of indicators) that gives red light green light type signals, with a high strike rate, is unrealistic.