System Development and Algorithmic Trading

Beginner Questions
1

Hi,

For those of you who don’t know me, my name is Clark. That is all. :slight_smile:

I am starting this thread as my way of contributing the little that I know about this subject as a way of saying thanks to the community as well as Babypips. If it wasn’t for the few individuals who offered their help, I don’t know where I would be as a trader right now.

If you want to know more about me and how I started as a trader here you go:
http://forums.babypips.com/q-corner/39183-featured-young-forex-traders-mays-picks.html#post260404

(It’s been a couple years since then and I’ve definitely grown up quite a bit, even reading those answers now makes me feel embarrassed)

[B]What is this thread going to talk about?[/B]
I’m starting this thread to share what I know in terms of developing systematic trading systems and how to properly test their validity in the markets. After the basics are cleared up I will go into detail about optimisation and explain walk forward testing. To wrap it all up, I will conclude with portfolio backtesting and optimisation. In further instalments I may or may not go into the technical aspects as to how to actually do what you need to do, the tools/software you made need and any other resources such as historical price data.

I am not much of a programmer (still learning), so there will be very little programming if any, but I can always direct you to resources that I have found that has helped me greatly. I will try to write as concise and detailed as I can, but I’ll apologize now as I’m not much of a writer or talker for that matter. To explain better I will do my best and provide diagrams and snippets that I can find lying around.

[B]In short, the topics I will try to cover are:[/B]

[ul]
[li]System Design
[/li][li]Requirements for System Development
[/li][li]Backtesting Trading Systems
[/li][li]Initial Optimisation of Trading Systems
[/li][li]Evaluation of Test Results
[/li][li]Factors Affecting Your Testing
[/li][li]Walk Forward Analysis
[/li][li]Risk and Trade Management, Position Sizing
[/li][li]Portfolio Optimisation and Strategies
[/li][li]From Theory to the Applied
[/li][/ul]

[B]What this thread is NOT:[/B]
[ul]
[li]Purchasing or Reviewing EAs (Expert Advisors)
[/li][li]Promoting EAs (Expert Advisors)
[/li][li]Programming (Unfortunately, I’m not the right person to talk about that, considering I hardly know the difference between a Boolean and Double :P)
[/li][/ul]

As a side note, 100% of what I’m about to share is NOT mine. I didn’t make this stuff up. I’ve just collected what I know and condense it all into one place to help future algo traders. So yes, you’ll see topics covered in books and other websites. Please give credit where its due.

Also, this thread is for [B]SERIOUS TRADERS[/B] and system developers. This isn’t about finding the “holy grail” through social trading mediums such as Zulu or buying some curve-fitted Expert Advisor online or any of that bullcr@p. Automated trading, like any other methodology takes time to learn and implement. At the same time it is resource heavy. so be patient. You’ll get it eventually. :wink:

In the future, I might talk about my personal ATS and its infrastructure and perhaps share some of my own experiences but for now I will talk about the core foundations.

That was my introduction.

I welcome all comments (and compliments :D) and it would be great if beginners and experienced traders contribute alike as I still consider myself a student. I have dedicated a good portion of my life already to this subject and I expect many more years to come (I am in Computational Finance in university, so I am kind of forced as well).

Without further ado, I will start in the coming posts.

Clark

2

Finally!
May the force be with you!

3

yep, now we can finally have a thread to discuss this stuff for real. excited to see what you put up clark.

4

I don’t speak for clark but, I’d say if your a skeptic or you feel your system is the bomb, remember this thread ain’t a pissing match arena, its a spot to learn and improve, however if you do feel the need and since most will treat it like a court case its not about what you can say its about what you can prove.

In the spirit of this thread

010010010010000001101100011011110110111101101011001000000110011001101111011100100111011101100001011100100110010000100000011101000110111100100000011101000110100001101001011100110010000001101100011001010111010001110011001000000110110101100001011010110110010100100000011011010110111101101110011001010111100100100001

(I look forward to this lets make money!) in short.

5

01000011011011110111010101101100011001000110111000100111011101000010000001100001011001110111001001100101011001010010000001101101011011110111001001100101001000010000110100001010

6

01001001 00100000 01100100 01101111 00100000 01101110 01101111 01110100 00100000 01110101 01101110 01100100 01100101 01110010 01110011 01110100 01100001 01101110 01100100 00100000 01100010 01101001 01101110 01100001 01110010 01111001 00100000 01100011 01101111 01100100 01100101

7

0100101101100101011001010111000000100000011101000110100001101001011100110010000001110101011100000010000001100001011011100110010000100000010010010010011101101100011011000010000001110011011101000110000101110010011101000010000001110111011100100110100101110100011010010110111001100111001000000110110101111001001000000111000001101111011100110111010001110011001000000110100101101110001000000110001001101001011011100110000101110010011110010010000001100001011100110010000001110111011001010110110001101100001011100010111000101110

8

0101010001101000011010010111001100100000011100000110110001100001011000110110010100100000011010010111001100100000011001100111010101101100011011000010000001101111011001100010000001110011011011010110000101110010011101000110000101110011011100110110010101110011001011100010111000101110

9

I’ll pitch in where I can:)

Stop fartin’ around, and git-r-rollin’ Clark!

10

Filtering and importing 1.4 billion lines of tick data right now… Computer is running a tad slow…

(I’ll talk about this ;))

11

subscribing

12

Sorry couldn’t wait. :stuck_out_tongue:
How do you test your system. Do you use software like Forex Tester 2?

AM EXCITED!

13

Nope! :wink:

And I’ll start ASAP. :slight_smile:

Thanks for the support everyone. Tang can be the programming side of this thread since he’s much more experienced than I.

Clark

14

really looking forward to this. maybe i can contribute a little every now and then too. i’ve done some empirical research as well. some for uni and some just for myself. i did it with matlab, but it seems that nobody else uses it for programming. so i probably wont say much about programming, but something about testing methodology.

cheers

15

If I’m the only one on the programming side, this is DOOMED:p

Hopefully people like CodeMeister, Goldylox, Shr1k, and others will help out. They know what they’re doing…
I ain’t dry behind my programming ears yet.

But if it’s something I’m comfortable in, I sure will:)

16

Automated trading is easy. Here’s the plan, it’s really quite simple: We each Google the best EAs for sale. Then we vote as a team on the best one and we buy it. Then following its precise instructions, we set it up on our MT4 platform. But here’s the trick, we double the risk so we make more than everyone else! Sounds simple right? It is! There, that’s my thread, the rest is just random rambling… :smiley:

Alright now that I’ve got your attention…

To start off, I just want to get everyone caught up and clear up some terms and definitions. This way we will be on the same page before going further into more interesting concepts. At the same time, I will talk about trading systems, otherwise known as “algorithms” (or algos for short).

I would like to point out that the concepts or terminology that I use may or may not be “academically correct/sound”. Sounding fancy and sophisticated is not our goal. Rather, we are trying to simplify and demystify this area of trading, so that individuals without a PhD in advanced mathematics can understand. It seems there is a large cloud surrounding this subject.

[B]What is a trading system?[/B]

Let’s start by giving a concise definition on what a “trading system”, “algorithm”, or “algo.” is in our context. Essentially, a trading system is a set of systematic, mechanical rules that clearly defines entries and exits in a given market/financial instrument. These rules can be defined mathematically and are exact, leaving no room for error or discretion. A trading system also incorporates specific risk management rules (stoploss, target profits, etc). As a result, these trading systems can be automated.

Technically, a trading system only includes the entry and exits, and a “trading strategy” is comprised of a trading system and risk management, but I don’t see a point in getting into technicals unless it’s for a research paper.

Note: [I]At this time I want to clarify on the differences between risk management versus money management. Risk management, as stated above, is comprised of the setting of stops and targets on a particular trade. Money management on the other hand, is focused on position sizing, how best to allocate one’s capital. Money management is essential in the construction of portfolios. [/I]

Because of the nature of these algorithms, there are advantages as well as disadvantages to using them, but I won’t get into them here as they are pretty self explanatory. (This will be everyone’s assignment ;))

[B]How do we develop trading systems?[/B]

The basic procedure in developing a trading system is quite simple, an individual generates a trading idea (it could just be that he/she notices a pattern or edge that may be exploitable in the markets). The algorithm is designed, then it is backtested using high quality, accurate price data, then optimised for its inputs. The algorithm is constantly being evaluated (using sound techniques) to ensure it will perform well in the future. Assuming backtests and optimisation all came out positive, the algorithm may now be traded live (or demo). But the work isn’t over yet. Algorithms need to be monitored for any signals of it losing it’s previous profitability, as well as periodic optimisation needs to be implemented in order to keep the algorithm up to date. It also serves as a tool to check the status of the algorithm. All of this will be discussed in great detail in further posts.

Now that we have an idea of what trading systems are we are now able to discuss just how one builds, test and apply them in the markets. In further posts I will talk about the techniques, processes as well as tools that one may need in order to accomplish such tasks. Perhaps, [B]we could even design a simple algorithm together in order to see the process first hand.[/B]

I know this stuff is a little basic and may be dry to the experienced algo trader, but be patient, the exciting stuff comes very soon now that the foundation has been laid. :slight_smile:

As a little conversation starter, let’s all brainstorm the advantages and disadvantages of implementing an automated trading system! Try and argue for both sides. Sometimes it is during these friendly debates where a “light bulb” moment occurs. :wink:

Clark

17

As for the little assignment, I’ll go first:

Advantages:

  • Do not need to watch the market 24/7
  • Emotions do not play a role in the trading anymore

Disadvantages:

  • Algorithm may lose its profitability
  • Takes lots of work!

Clark :slight_smile:

18

Advantages:
-Optimization

Disadvantages:
-Optimization

19

I can definitely relate to this!

20

There are many many many talented programmers out there who do nothing but coding all day. Some of them even offer to code other people’s ideas for free, hoping they will discover some new idea themselves. What the above should tell us is that designing a successful trading algorithm is not easy at all. The biggest problem of all is of course consistent performance year after year. Almost all the EAs I have designed have been successful for a few consecutive months or even a few consecutive years, but I still haven’t managed to program something that has been successful for a decade or more.

The thing is that a few years or even months may be enough, assuming that you get to live trade that EA at the start of its good period. Unfortunately you can’t know beforehand when the market conditions will change.

In any case, just think that there are tons of programmers who design, promote and sell EAs all day long. Their main focus is on programming, not trading. And they’ve been doing this for years. If even one of their own EAs would work and give an average of 5-10% per month, then they wouldn’t need to sell EAs anymore. But instead they actively promote their EAs and enter competitions with prizes of a few thousand dollars and build fancy websites promising the wealth of your dreams.

What I’ve been trying to say with the above is that if your idea is relatively simple, then it has already been coded and it is not working in the long term. If it were, then you wouldn’t be seeing so many EAs for sale and so many people who code indicators and EAs for a living instead of trading for a living.