Fourier analysis ~ has anyone ever done such a thing with respect to forex?

Beginner Questions
1

Reason I’m asking, is that I don’t want to waste time duplicating work. I’ve googled a bit, but nothing I can find is terribly clear, that I can see.

For anyone unfamiliar: imagine that any waveform (such as a price chart) is a sum of component frequencies, each with various phases and amplitudes which can be described with a function. For instance, if the market were a pure sinusoid, or maybe a sum of two sinusoids of fixed magnitude, the market would be flawlessly predictable and everyone would be rich (or, break even, as no one would lose money ever again).

Of course we aren’t dealing with anything nearly so perfect, but over short timeframes and with some attention paid to the periodicity of one day and lower, any ‘edge’ to be had will stick out like a sore thumb, such as a huge dip or ‘spike’ at certain frequencies, when compared to others. And yes, I know that markets are supposed to be fractal and so forth, but as many of us know: asian session often doesn’t have the range that London does, &c &c. True fractals don’t have such limitations, thus, there are some basic statistical variations out there. Maybe nothing big, but I’d like to have a look at them anyway. A 51%/49% bias is more than enough, if it’s robust.

There are some websites with rudimentary analysis regarding seasonable variability (one particularly long, singular frequency component of 52 calendar weeks); I am sure there are some hard stats with regard to weekly, daily as well. I’m wondering if anyone’s compiled this sort of thing anywhere, or, perhaps just in private? If I had 20 or 30 years of tick data in the right format, and say, a friend with a Mathematica licence, it would be fairly easy to have a look… but I don’t quite have that. But I’m sure someone must have done it. Something a bit more graphically and completely presented than “here are the daily ranges for typical Monday through Friday, and the weekly pivot points.”

Eremarket has mentioned a statistically significant bias toward S1/R1 turning points, and ICT has mentioned that maximums or minimums often happen in the first four hours of trading… perhaps we can get a graph of all this sort of thing combined?

That’s what I’m looking for with Fourier analysis, though it would have to be ‘back translated’ into useful statistics from there, I’m sure. The idea being, one could also see how robust such stats were with regard to each other ~ loose correlations, strong correlations or what. For instance, if Statistic A implies price may go up and Statistic B implies it may go down… which to weigh more?

Sounds a bit abstract, and yes it won’t be able to process individual events like ‘Bernanke started talking pessimistically and price did X’ but over the long haul, may expose some advantages.

Insights very welcome. This is for my longer term forex education. And of course if anyone happens to have price data and say Mathematica or MatLab I’d love to see some results…

2

This one?

Extrapolator - MQL4 Code Base

If you are looking for more, just put “fourier indicator mq4” into your Google search field" :wink:

Nice idea btw. More for my spaghetti bowl, lol.

3

Sort of like that, but… radically different, if such an explanation makes sense! They did do a fourier fit of the average price curve there, but… I suspect that is misapplied; the ‘correlation is not causation, and hence not predictive’ issue.

Rather, I’d approach it like this:

  1. Start with 10 or 20 years of price data, ideally tick data, with weekends removed such that it was all business days (ignoring national holidays of this or that country, probably)

  2. Graph price, derivative of price, and second derivative of price with respect to frequency (1/period)

  3. This is from a very unrelated field, but the data output would vaguely resemble this:


Note that pure noise has no significant spikes, whereas a reliable trade strategy would show itself as a peak such as in the periodicity shown in the second row… or even as shown in the third row. The stronger the correlation, the clearer it is.

Further, there’s such a thing as trending markets, markets aren’t dead flat like that… so it’s important to usefully knock out the low frequency stuff in order to properly have a look at the higher frequencies at lower amplitudes ~ this can be done with a bit of intelligent filtering. Same thing for when you want to look at long period, low frequency stuff: a sensible high frequency filter can be applied to the data… this is vaguely analogous to setting an intelligent stop loss per length of time expected to be in a trade.

In fact, the combination of the frequency/periodicity, the frequency filters used, and strength of correlation should give a very clear description of actual statistical edge over very long time periods.

Due to the correlation~is~not~causation bit, such analysis is simply a measure of what the odds are, and won’t tell you what has to be coming down the price curve in a few minutes. Just what its statistical likelihood in the past has been. That’s why I think the indicators that have been done so far are misapplied.

We would know if a fourier analysis was done right, if it correctly reflected the lack of volatility during asian session on the 1/day frequency, for instance. Not a lot to be gained from that particular insight, but it would serve to prove that the rest of the data was analysed ‘right’ with respect to cyclic recurring probabilities. Of course, markets can change, but anyone who uses any information prior to what price is doing at the current instant in time buys into ‘past market data can be somewhat predictive of the future’…

To really take this to an advanced level, it may be possible to categorise trading days into a few more narrowly defined types, and run fourier analysis on each type… you’d get multiple correlations, but maybe “Only Mondays data” would give you a much clearer picture of what was likely. Of course you would lose all frequencies greater than 1/day doing that, but it still could be useful.

I’m sure someone has looked at this carefully this way in the past; this is a very, very standard scientific analysis method. The results may be solidly useful, or they may show it’s nearly totally random, but whatever the results are, they will be clear. It’s just a tedious matter of preparing, then feeding the tick data into Matlab or Mathematica or suchlike… even Excel could probably do it, though it might tie up Excel for a few weeks chewing the data.

Put it this way: if whomever has done it does not spill their data, eventually I’ll run it, and squawk like a parrot here with blindingly clear analysis that even a remedial~math high school student could understand. :slight_smile:

4

Desmond,

I won’t go into the details, because I have not coded that indi. However, it seems to be a correlation model. If you have enough data, you can for sure use it on the past 20 years.

However, it looks pretty good. I just took a S&P weekly chart. Changed Lastbar to 50 and Harmno to 10 because I guess 20 is a little overdone and everything else as standard. Now look at the chart. It predicted with a pretty good accuracy the price after last years November (white line before and blue after it is the prediction). Sure it let out the big drop this year, but the tendency is down after going up in the first half of the year. You can experiment with this indi with different values. Imho a nice tool to add for any trader.

Screenshot - 11192011 - 11:08:01 PM.jpg700×452 50.7 KB

5

Here is one with eurusd, monthly. Looks pretty neat imho. A blue ma of 50 to compare it with.


6

Here is an m5 on eurusd:

Screenshot - 11202011 - 12:01:01 AM.jpg686×700 29.4 KB

Really, whatever I randomly check, it is at least with the first 10 prediction bars pretty accurate. Plus in many cases even with direction right for the next 50 bars.

7

I would love to see your results if you pursue the FFT, and if you’re willing to share. Should be “fairly” straight forward. If you use MatLab, look for the sun spot example.

Rich

8

Interesting…

It may be, in fact, right for reasons I don’t fully understand… but it’s a really different way of looking at things than I’d expected, and I’d not expect such a thing to work, offhand. I’d be extremely cautious.

I had to pull an unrelated graph to describe what I’m getting at ~ here’s an example:

edu05_0018i1-en.gif468×581 6.74 KB

Here you see a graph that has two clear frequency components:

  1. a loooong slow rise and fall
  2. a more rapid ‘up and down’ within that

Both of (1) and (2) would appear as ‘spikes’ on a fourier spectral analysis graph, with the rapid rise and fall (2) being the far stronger spike. Lots and lots of data = better result. I’d simply graph this and show it, but, I don’t have Matlab here. It’s something I’ve worked with extensively, but last I checked it was just ridiculous money to buy it ~ a couple thousand not counting any add ons.

In our case, whatever the trade data might be, there would be some spectral signature simply due to the Sydney/Tokyo/London/NY variances in volatility ~ those in particular may not be terribly useful, but, some other things might pop out along with it. I think it was ICT that said daily highs and lows were typically seen during the first four hours of trading; things like that, in fact all long term statistical ‘edges’ and their relative strengths and reliability would be exposed with a solid fourier analysis.

Of course there are some data ‘artifacts’ which commonly result that might be worth ignoring (akin to overfitting) ~ but if you run such an analysis on a few more mundane, non trade related things you can quickly get a sense of what is significant and what isn’t.

And again, even this may not be predictively valid… but just a snapshot of that past market performance actually was, with clear biases highlighted for anything the market did, or didn’t do over the sample period.

Just food for thought… I’m sure a few people here are maths or stats experts, perhaps they would chime in?

9

I’d simply graph this and show it, but, I don’t have Matlab here. It’s something I’ve worked with extensively, but last I checked it was just ridiculous money to buy it ~ a couple thousand not counting any add ons.

For those that use MatLab at work. Look into Octave! It’s open source (free) and cross platform. It’s very very similar to MatLab. QtOctave is the GUI. Nearly 100% compatible m files with MatLab.

10

I am neither, but there are not enough data points for either the long term(10 year) or short term(1 year). This is getting into the Nyquist theorm or anti-aliasing effect.

But in regards to the market, data points for each open day would be sufficient to extract FFT information for a year or multiple decades.

11

In trading (as in every aspect of life) there are two components at work. Theory and practise. I’d not go too much with the first part. Market is built with random, fractal and trend components. The trend components are imho responsible for the waves. The issue with spectral analysis arises if you try to find out how many significant waves there are. Plus even if you find out the precise number, the market has still components of random and fractal data. In my chart of the S&P above one can see a pretty nice accuracy of such an approach, predicting future outcome of past waves. Not going to the detail as I said I have not coded the indi, it is clearly to see that it predicted one year in advance the upmove of the S&P in the first half this year and the downmove the second half.

However, what is imho almost unpredictable by this approach is the precise pip value where it moves down or up to. The harsh drop of the S&P was deeper than what was expected by the wave prediction. That is what the market does. That’s why I said it might be a nice tool in addition to others. I guess if you start extending fourier analysys approaches to a holy grail, you will get nothing but losses. As with all other approaches in isolation. What makes trading successful is the stacking of odds in your favor. You can use that prediction with regression analysis, pivots or whatever and then it might become a system with an edge.

It is even good to know that the market doesn’t follow to a hundred percent the statistical rules.

Successful trading itself is simple. Don’t overcomplicate it!

12

Very well said.

K.I.S.S.!

13

Great input on this thread; indeed there are limitations (Nyquist and a few other issues) and also sufficiencies (you can be good at trading without ever knowing this stuff).

I guess what I’m going for is this: it’s like the tale of the blind men and the elephant. Each may understand the elephant ~ one knowing the trunk, another the tail, but who understands the approximate shape of the whole elephant? And, can you know if you do or not? That’s sort of what I’m going for here: a snapshot of the entire statistical elephant, at least from a long term standpoint. There’s a lot it won’t show… for instance it would be nearly impossible to tease out common sense support and resistance principles, and so forth from it. [I]But if I were able to cook up a few spectral analysis graphs with regard to price, its first derivative (akin to momentum) and perhaps price ‘acceleration’… would anyone else here like to see it? [/I] grin…

It sounds like such an incredibly standard analysis hasn’t ever been done and shown to the forex ‘public’… if so, this really needs to be remedied. I strongly suspect there are a few clearly tradeable dips and spikes in there, if only people were to see them. This is data analysis 101, really… perhaps not wholly predictive, but we’d definitely see what the market has or has not done.

Perhaps it boils down to obtaining this Octave program, and then tick data in a format that can be easily fed to it. More critically, interpreting the data in a sensible way… for instance, noise can be wrongly interpreted as ‘a whole bunch of really good spikes in the data to investigate!’ when it’s actually just… noise. There are signal/noise ratio filters that can be applied to data, with logical rationales backing up how and why… also allowing the opportunity to logically ‘shoot down’ such filtering if it’s incorrectly done.

I’m up for taking this on, the only really icky part is getting the tick data, then converting the data into a palatable format for… Octave, or anything capable of processing the data. I’ve done more than my fair share of that sort of thing, in the days of Borland Pascal and blinking cursors, but while I still can, nowadays it’s about as enjoyable an experience for me as tuning Skinners Union carburetters on a Jaguar (anyone who’s done that will know exactly how fun that is…)

If anyone is a computer expert willing to obtain such data and convert it… say just EURUSD and GBPUSD for side by side comparison, I’m happy to provide whatever fourier analysis basics I can. It’s been a few years since I’ve needed to do anything like that, but, it’s really little more than a few equations, a few filters, put the data in and press the button. It would also make a really unique bit of data for this site, as I can’t find anyone anywhere who has it posted… perhaps someone has google~fu stronger than mine?

14

The idea is a great one, DS.

The reason I wouldn’t go any further than to use this indi there is this. You can’t predict the outcome of pips. You can at best predict the outcome of direction in a specific time. The prices won’t repeat itself in detail. Just in broad averages. That’s also where statistics can fit it. Albeit as I said. If it gets too complicated, you probably have harder work than to be a university professor and don’t get more money out of it. If at all. If trading was a matter of fourier analysis and nothing else, the worlds richest people would all be professors.

The markets change and stay the same all together. In detail, markets change. That’s why it makes no sense imho to use tick data for 20 years to find details in it. Those details get rubbed away after time. What stays is the ever coming and going of waves. Not one wave like the other, but close.

Obvioulsy fa could be used to predict in a way the direction. To trade that profitably, you need other indicators or pa or whatever and then build a system around it. With detailed rules about sl and top. The “now” is more important than what was going on 10 years ago.

The eurusd 200 daily atr from 2 years ago was ~ 130 pips a day and now it’s 164. :wink:

Not to say I am right and that’s it. Just my opinion right now. Prove me wrong, please! :slight_smile:

15

At 4pm cet around today it predicted successfully the upmove in the fiber. Breakout.

Screenshot - 11212011 - 08:05:53 PM.jpg700×642 30.4 KB

After the second top it predicted correctly the down waves.

Screenshot - 11212011 - 08:07:17 PM.jpg700×646 31 KB

As I said, it’s probably not doing it all the time, but imho worth to spend some time with observation about the accuracy.

How I see it is a little like a pretty good predictor of a moving average. Just the highs and lows around the predicted ma’s are not certain.

16

Here is one where it predicted the upmove and the correction.

Screenshot - 11212011 - 08:22:55 PM.jpg700×645 30.4 KB

17

That’s really interesting…

…there are a lot of things that I don’t understand (obviously) ~ I’m not quite sure what rationale there could possibly be for a fourier curve fit going forward (as opposed to fourier spectral analysis of past activity) to apply to markets… but if it does, great! It would be a ridiculously powerful tool if it works. That’s a real headscratcher for me with regard to the ‘why’ of it, though. I’m not sure if I could personally make a case for or against. I suppose theoretically it may be part of some trillionaire bank’s trading algorithm or something.

Like you’ve said many times Buckscoder, it’s the drawdown that’s most important to worry about… I wonder just how often such a fourier predictive curve would be wrong, instead of right? Even if we only consider direction and not magnitudes.

With respect to the larger discussion, I do agree more recent market behaviour is more valuable than old behaviour (seems common sense). Quite a lot to think about!

18

Another one with correct prediction of the drop, the upmove and the correction. 50 candles or 750 minutes in advance! :o

Screenshot - 11212011 - 08:36:54 PM.jpg700×643 30.3 KB

19

Yea, that’s why I have this indi just on my radar now. Or better said in my toolbox. One needs to get a feeling for that. Same like with any indicator. If you believe it could be a good tool then just add it too, play a little with the settings and look in which particular cases and tfs it could be of help together with some other tools. :slight_smile:

It won’t likely be right all the the time, but even if it’s just right 50% the time then together with another tool of another dimension this would bring the edge in your favor by amazing percentages! :wink:

At least it all goes with the settings. How much candles for analyzing (300 are standard there), how many frequencies etc.

20

Regarding how it works I guess(!) it just looks to find the primary frequencies of some waves in the past by anayzing and tries to mix and predict it in the near future with the assumption the frequencies get not distorted. That’s pretty much the best approach you can have for such a solution imho. Sure it will get distorted by whatever random or fractal data, but that’s exactly the part nobody can predict. However, the less distortion by news, panic, etc. the better the prediction should be imho.