Simba… Nice response!
“Its perhaps rather unfair of me to comment on the specifics, as I have not purchased or used this particular EA, nor do I know anone who has.”
I agree - I did purchase the EAs and know several others via other forums so I have some skin in the game and probably why I feel the urge to fight the corner for these guys.
“However, what I would say, from personal experience as a system developer, is that the role of random chance plays a far larger part than we are perhaps prepared to acknowledge.”
I think you’re right - however hard you try, the market will bite you in the rear end iven a chance.
“In the case of these particular EA’s I’d also suggest that a year of backtesting might possibly be an insufficient sample size, and given the availability of historical data, there’s no particularly good reason not to test over a longer term.”
Also true to an extent, however, looking at how the market has changed behavior over the last 3 years makes me feel that any EA has a shelf life and that trying to create an EA that can trade uniformly over a very long test period is bordering on a Grail quest - the only one I’ve seen which seems to have achieved that while still producing good results is Megadroid.
“I’ve personally witnessed mechanical day trading systems that have performed well for periods of perhaps 3-4 years, over a sample size in excess of 2000 trades just completely fall apart. Although these systems may have looked to be performing well over relatively long periods and large sample sizes, in reality, these system’s did not have a positive expectancy over a longer time horizon. Its quite common, I’ve developed hundreds of strategies that initially look great, but subject them to reasonable analysis, and they fall apart.”
…but taken in context, probably most of those strategies would have traded well for a few months, after which they could have been “retired”.
“I trade with a high degree of diversification, and I quite often see substantial differences in performance from month to month and year to year, and although a major aspect of my job is to attempt to determine why this might be the case, I often conclude that these differences are often just due to luck.”
I personally trade a contrarian stance and long ago gave up trying to determine why the market does anything - it will do whatever it wants whenever it wants and any attempt to pick tops & bottoms is a game for chumps as the only guaranteed market direction is east!
“The issue is that you cannot really differentiate between the role of good and bad luck over such a small sample size. Unfortunately, the effects of random chance is something that traders need to deal with (and even possibly exploit to their advantage). The worrying aspect of course is that a run of bad luck could have such a significant impact on results.”
That applies to any trading system (manual or automated) - the only advantage of an automated approach is that rules will be obeyed.
“I have been quite critical of the vendor behind this idea for a number of reasons. I’ve touched on this point before, but a trading competition isnt perhaps the best vehicle by which to evaluate an automated trading strategy. By their very nature, competitions tend to encourage participants to submit overly aggressive strategies. A realistic well designed system producing realistic returns isnt going to stand any kind of a chance in the short term against an over optimised strategy that happens to get lucky.”
That’s true but the aim was to encourage traders with working strategies to submit them. Of course, some people (possibly the majority) will try to throw some overly optimized strategies together in the hope of taking home a prize but, as it happens, at least the #1 & #2 positions were grabbed by people who personally trade their strategies in one form or another.
“Anyone with a decent system is going to be protective of their intellectual property, and of course, theyre hardly likely to risk disclosure of a method given that they have little realistic chance of beating an agressive EA who just happened to get lucky.”
I don’t agree with you here - you only have to look through the various forums to find some real gems that have been freely shared and, as far as I can see, no-one involved in the FRWC competition was really risking the disclosure of their strategy to the extent that it could become a problem.
“I was looking at the myfxbook site earlier today who are currently promoting a possible future competion, and to paraphrase, the sales pitch is enter our competition “trade a demo account risk free, for real cash prizes, you have nothing to lose and everything to gain”. That kind of sums up most of the industry really, a bunch of people playing a game for fun, no different to any other game of chance. If you really serious about providing a customer with a well designed product, should you really solicit for potential products through a competion ?”
Well, considering the quality and instability of the crapbots launched on a weekly basis, I’m starting to think that the competition approach isn’t a bad way to go!
“Lets assume that the vendors of these products are actually completely ignorant of the issues involved in system design, and that they just had a badly thought through idea that the past performance of a system was an indication of future potential, kind of like betting on the leading horse half way through a race. I’d ask the obvious question why not subject participants to more stringent backtesting. Why not insist on a 10 year backtest, why not insist on montecarlo simulation etc ?”
As I said before, market conditions have fluctuated drastically just within the last 2-3 years so a strategy that works well in 2002 is almost certainly going to fail in 2010 and vice versa.
“The whole basis of picking a winner by allowing a bunch of people to go head to head in a competition, where risk is ultimately rewarded rather than sound strategy design really doesnt sit right with me, but I could at a push, believe that these ommissions and decisions where taken out of ignorance.”
The FRWC guys released 11 EAs, all of which traded with different levels of risk and aggression so purchasers had plenty of scope to select a suitable EA or two from the mix and match the strategy to their own risk profile. Unfortunately, most people seem to have focussed solely on Fusion which is an aggressive EA whichever way you look at it - certainly not an EA that I’d dare to place on a $1,000 account unless nano-lots were available.
"As for negligence, lets assume that I decide that I’m going go into business building boats, hey, rich people buy boats right ? they cant be that difficult to build can they ?. I build and sell a few boats, then one sinks, drowning the crew. I’d certainly categorise my behaviour as negligence. Its not really a suitable excuse that I thought I knew what I was doing, or that I’d built a boat, and sailed it around for a few months in calm sea’s without any issues.
The same goes for the vendor of these EA’s, they released a product without appropriate testing, and as a consequece, punters possibly suffered financial losses."
Quite true except that the scenario doesn’t apply here… the EAs were tested reasonably but, in my view, users simply traded too aggressively given the size of their accounts. Further, many people immediately threw the EAs onto live accounts when they should have trialled thingson demo accounts for a while to get more of a feel for how individual strategies affected things. Note that I said “get a feel” rather than “test” because I don’t believe that any amount of testing really makes much (if any) difference and neither does “experience” or the lack of… the WhiteStar line were very experienced ship operators and the Titanic was exceptionally well designed and built, having passed many tests, but it still sank!
“Of course, losses and drawdowns are to be expected, and one might expect that a responsible vendor would stipulate a reasonable estimation of the levels that a potential purchaser could realistically experience , and to specify the point at which they considered that these systems where no longer operating within the parameters in which they where designed to operate. However, the marketing by this particular company appears to be promoting the potential rates of return, based on extremely dubious short term testing rather than focussing on risk.”
Marketing is marketing and I don’t think they did anything out of the ordinary in that respect. Look at the long-term testing that was provided for FAP Turbo - 100% per month gains that went back ages but, after the first few weeks, nobody has seen those kinds of results since.
“I fully understand the need to show a product in its best light, and the constraints under which they need to operate to realistically make sales, but I would suggest that to allow their customers to endure such high levels of drawdown that presumably exceed levels experienced in previous testing is actually negligence.”
The FRWC didn’t have control over the individual EAs but they did provide updated versions and recommended changes to the EA settings that were designed to stabilize things and reduce potential losses. Suggesting that they acted negligently is akin to blaming Ford with negligence because some people drove their cars while drunk and sustained serious injury - the vendor/manufacturer cannot control how owners use their products.
“Unfortunately, you have to consider the possibility of deliberate deception, I personally know several individuals who run a whole series of forex related businesses that are built entirely on deception. They do so because its easy, and its reletively lucrative. I’m not suggesting this is the case, but I’m stating that it happens, and it shouldnt be discounted.”
I’m with you 100% here and, interestingly, so are the FRWC guys - perhaps you didn’t read their 40 page expose (a very interesting read!) but they really went to town and highlighted many of the tricks and deceptions used by the outright scammers that you’re talking about.
“An associate of mine sells an EA, and he provides backtests. Initially he engineered the backtest to run over a time period conducive to the EA’s performance, ignoring the 20 or so years worth of data that didnt really give the result he required. Is that deception ?, he argues that markets have changed, the EA works in current conditions. Its a fine line.”
It is, but it’s also very valid.
“Now of course he (and countless others) have modified EA’s to exclude specific periods of time so if a customer attempts a backtest, the EA skips periods in which it does less well, I’d argue thats deliberate deception. I’m not implying that its been done in this case as there appears to be a complete absense of backtesting, but the point is just becuse someone provides a backtest does not necessarily mean that everything is above board.”
Absolutely true - I’ve seen several EAs with “black-out” dates to improve backtests but I don’t believe that is the case with the FRWC EAs - actually, they have a clause in their rules which specifically forbids the practice.
“As for refunds, it seams perfectly acceptable to me that with a digital product that the vendor should place some sort of constraints on the circumstances in which a refund can be requested. 30 days isnt a particularly stringent requirement to impose on purchasers.”
I agree.
“The problem that I would forsee is that the strategy employed by their sales and marketing team possibly sets unrealistic expectations on potential purchasers, and that in turn could lead to an increase in the number of people requesting refunds, and potentially resorting to chargeback requests if refunds are not handled efficiently, and that is a genuine risk that the vendor will need to manage.”
Some people will always try to buck the trend / bend rules - it’s human nature but I know that the FRWC have been very flexible over what constitues the “30 days trading records” - I’ve heard about some pretty interesting mixes of live and demo account histories that have been accepted to make up those 30 days.
My personal view is that anyone who instigates a chargeback when the vendor has a reasonable refund policy is nothing but a fraudster and/or a malicious individual so the sooner that the likes of ClickBank and Plimus follow PayPal’s lead by contesting chargebacks (rather than simply rolling over), the better it will be for the honest customers out there.
I have a friend who sells via ClickBank and we all know that CB will issue a refund for no better reason than your coffee got cold before you managed to drink it, yet he still gets hit with chargebacks (and the associated unrecoverable bank charges) from morons who must be too stupid or vindictive to ask for a refund like normal folk!
Neo
