Statistics in Technical Analysis

I wouldn’t advise to try to calculate probabilities here. It is much more logical to define and backtest a new indicator C = enters when both A and B enter.

Formulas I posted are correct, but now I realize one can’t even use them here because of how (tricky) these events are defined. These events actually consist of 2 probabilities 1) probability of entry signal 2) probability that entry signall will be successful.

Actual probability (success of indicator C) intuitevly for me could be anywhere from 0% to 100%.

Hi guys, I would like to test the Market Structure concept, and see how many subsequent Highs/Lows we get after a break in Market Structure. And I want to visualize it with a bar chart about how many times we get 4 sequent higher highs, or 5 or 6 etc…

I feel there is a difference in Lower lows than Highes, so I will try to split the results…

So before I begin, does anybody have a tips about how to start, or things to consider when running the experients, comments etc?

PS. I have problems understanding Probability too, so if anyone have a good beginners resource for probability, please share :slight_smile:

Perhaps you could map out the instances where there have been two successive higher highs then record if there is a third.

So on a chart you may see a lower high, then a higher high, then another higher high. This would then be the pattern you were looking for, then tabulate the result if the structure breaks continued or not. If you repeated this then with examples where there were then three breaks upwards, then four breaks upwards etc you would be able to calculate the percentage chance of the next swing’s direction.

You could calculate the probability by dividing the number of continuing structure breaks by the total number of patterns you looked at. For example if you looked at 40 patterns and 10 broke upwards 10/40 would give you 0.25 or a 25% chance.

In terms of reading up on statistical analysis I can’t recommend anything beyond online media and presentations of lectures. I would be interested in books with studies done on trading strategies though.

Hi guys, i have gone true the data, and have attached a csv file(you can open it in excel/google spreadsheet etc) and it contain 4h candles from AUS/USD, and contains 4 extra rows that has true if it is a low/high or ITL/ITH…

Ill try to do something useful with it now :slight_smile:

(hope its okay Im spamming your thread Woloo)

THis is an EXCELLENT thread. I will contribute to it soon

I had a play about with pivot points and wanted to see how often they acted as support and resistance. I looked through roughly six months worth of data starting in may 2010 for my samples.

I was interested in finding out how the GMT pivot points operate as support and resistance by themselves.

So I set up a simple system of rules for this rough test. To classify a bounce I am using two hourly candles closing on the side of the bounce. A break through will be counted if there is a break through of two hourly candles. My plan being to see if there were biases for where price would move following an interaction with one of these levels.

The results:

What we can see in this image are the values and percentages for the direction price was moving from towards the pivots (Above Bellow) the percentages of time price hit the pivot and it acted as support or resistance and the percentages for the number of times price broke through.

The percentages are calculated relative to the side price was coming from. So the percentage for support is calculated from the number of time the level acted as support divided by the number of times it acted as support + price broke through.


Looking at the levels just as support and resistance there are roughly equal odds for price to break through a level or to react at it.

It’s also interesting to look at what happens when price does break though a level. If we look at S1 price has broken down 32 times out of 60. Yet in those 32 times only 12 (37%) times does price return to S1. This is seen with R1, S2 and R2 as well. This suggests to me that as price moves beyond these levels it is less probable for them to return to them let alone break back through them. Note the percentage for the times that S1 acted as resistance and R1 acted as support.

Anyway it’s all food for thought, I wanted to give it a go after reading though Homeofgolfs pivot post. Because that revealed some really cool things.

Feel free to post what ever you want, I made the thread as a place for people to discuss and try to understand how and why we use the tools we do from a probability perspective. I’ve never really fully understood the tools I’ve used to trade with beyond being told or reading how to use them. It’s interesting to look at them from this angle.

With all the talk of higher highs and lower lows and pivots and such I thought I would make another addition to the wonderful world of pivot points and just how useful/useless they are.

I have noticed in the past from casual observation that plotting daily historic pivot points (with 5pm est calculation) acts as something of an advance moving average. I noticed that price has a general tendency to stay above the pivot if today’s pivot is above the previous day’s pivot. I have taken that knowledge as a given since I have observed it with my own eyes. In the words of Obi Wan Kenobi, “You’re eyes can deceive you. Don’t trust them.” So, I did a quick test to see if this was, in fact, the case.

If a day’s pivot is higher than the previous day’s pivot then the average daily price (H+L)/2 should be greater than that day’s pivot point. So, is this true? Yes.

It is not true in a fall to your knees weeping with the sudden revelation that you have discovered the keys to a vault which contains a source of ultimate power kind of way but it is true, nonetheless.

EURUSD
Sample: Past 900 days
Pivot Point opened above previous day’s pivot point: 463 Days
Daily Average Price was above pivot point: 283 Days
Percentage: 61%
Pivot Point opened below previous day’s pivot point: 437 Days
Daily Average Price was below pivot point: 277 Days
Percentage: 63%

Just to compare I also checked AUDUSD over the same time period.

AUDUSD
Sample: Past 900 days
Pivot Point opened above previous day’s pivot point: 482 Days
Daily Average Price was above pivot point: 287 Days
Percentage: 59.5%
Pivot Point opened below previous day’s pivot point: 418 Days
Daily Average Price was below pivot point: 255 Days
Percentage: 61%

The AUDUSD’s performance was just a bit lower than EURUSD but there is definitely a general tendency for price to remain above the daily pivot if it opens higher than the previous pivot. If the daily pivot opens below the previous day’s pivot then price will tend to stay below the pivot.

Useful or not? I guess that is up to you. Keep in mind that pivots are calculated in advance for the next trading day :15:


This cant be correct with a moments logic

if the probability of throwing a six is 1 out of 6, which it is, forget convering to percentage, it is 1/6.

If you throw the die 6 times, and ADD:

1/6 +1/6+1/6…

you get 6/6

=1

= certainty!

This is a good start guys, I do this analysis for every tool in my toolbox. Although previously had shared information it never really stuck, although it was mostly in the chat room. Most people just glazed over it. I have 1 problem some of these posts with the manual back testing, its not a repeatable process for everyone because its inherently subjective. So I can not verify your work, even given the same data set. When I do my research its 100% programmed so that its is a repeatable experiment for anyone, this is what is required of scientific findings. Unfortunately most of my research is on futures contracts, if you guys are still interested I will share them. I hate to criticize then contribute nothing. But if this thread is going to take off, i think we should have a set of rules so all findings meet some minimum criteria in order to be seen as valid.

Yes, and there was a post with pivots from 2 hr. chart. Thats great for software that allows 2 hr charts, but Metatrader jumps from 1 to 4 hr, so making the application limited.

However, in the spirit of contribution, heres a book everyone might want to check out:

Microsoft Excel for Stock and Options Traders, by Jeff Augen.

All of my stuff came from using Excel and should be repeatable by anyone who cares to crunch a few numbers. In other words, no subjectivity (except for my intepretation of the results which is as subjective as it gets). Variations will exist due to using different brokers or, in the case of pivots, using different calc times. The general trends should, however, hold true if the same conditions are applied (5pm est for close of day and pivot calculations).

Futures data will also have variations from the spot markets but general trends should, hopefully, apply across related markets. I would love to see anything you have tested in the past.

Well I thought I would contribute something to this thread. this is an update from my original thread which caught little traction but maybe this is the right crowd. Some of the jpegs are dead now from the hosting site i used so i attached a txt file with all the stats.

http://forums.babypips.com/newbie-island/46361-datamining-quantifying-your-markets-personality.html

I do this type of very basic data mining on every instrument I trade. The data feed I am using is FXCM. The dates for each level of study are noted for each time frame, all bars are built using 1 minute time bars.

This study is based on the closing prices of bars, I am using % ages as opposed to pure price movements for several reasons. Usually at higher prices we have higher volatility, changing to percentages helps remove that bias. Also financial time series are in general non stationary and non normal, so determining any confidence level becomes more difficult, changing to % returns (you can do log returns as well) creates a weak stationary time series which becomes easier and more reliable to work with.

lets address what each field means.

of periods = how many bars of that time frame were analyzed

of Moves = how many bars consecutively closed in the same direction

Average period in move = how long they lasted
Avg AMP = the amplitude in % the moves traveled
Chance to last X+ periods = is the % chance that a move lasts X or more periods/candles
Chance > X % move = the % chance that a move larger than X percent may occur.

My analysis of this data:
Euro/Usd is symmetrical in both directions on all time frames, this is not the case with other instruments. Take stocks for example they exhibit a long side bias. This is both for the num of moves as well as the average amplitude of moves.

Also note that as the time frame decreases the average amp (volatility) does not decrease at the same rate, it decreases significantly less. So you would expect from the daily to 4 hour ( 6 periods) a 1/6th the average amplitude of the daily move (.155) you actually get (.37) about double what we would have guessed in a linear relationship. This is true again for all time frames.

Lets look at the characteristics of the % chance of a X + move, think of this as a trending characteristic. The % chance between the daily, 4h and 1h are all very similar, only 1-2 % age points difference. Now lets compare daily and 5 minute. Most people would say that the higher the time frame the less noise, and the “easier to trade”. So this comparison would be natural as the 5 minute is very “noisy” as seen on numerous forum posts. The 5 minute % chance of X move is larger than the daily % chance of X move in every category. Actually its larger than every chance of X move on every time frame. But the significance comes from the chance for a 4 + period move. About 6 % (5.81 % on average) larger than the daily time frame, it is 3.79 % higher on average than the hourly time frame. What does this tell us? That lower time frames have “momentum”. That they tend to exhibit more trending behavior than they are given credit for, however the amplitude is very small so the amount of market friction (slippage/commissions, execution latency etc.) becomes a larger factor. But this just shows that being profitable at the 5 minute and lower level is possible and there is an inherent edge there. This edge is best shown by the 2 + % chance stat. taking a random entry (50/50 long short) the % chance to last 2 periods respectively is 51.96 and 52.04. Basically a net 2 % edge either direction. Now is the bar large enough for a retail trader to profit due to those market frictions, probably not. But you can see where the HFTs and large institutions have a fertile trading ground.

Great. Thanks to MeiHua, any future posts I make on this thread will have to be worded so it sounds like I know what I am doing (which is doubtful). Interesting stuff Mei. Seeing market truisms refuted and dashed upon the rocks like that is kind of fun to watch.

I hate to see this thread die. Especially after my post. I truly believe in quantitative research and evidence based TA. I don’t know if it’s just this community isn’t ready for this type of material or just plainly isn’t interested. I was planning on doing more post as time went on and discussing others findings. But there hasn’t been anymore shared. I could post a some of my work here but I feel it would high jack the thread if I was the only one sharing . So if there are people still interested in that kind of work I may split off in my own thread just for that or I hope to see more people share here. Like I said in my original post about this my thread and this type of topic always die. If anyone can give me insight into why that would be great. Is there truly no interest in scientifically repeatable and statistically significant results???

I am interested, but the problem is that to post on this thread, I would expect myself to have something insighful to add that includes research I’d done, which I can only do at a minimum. It is a really good thread idea, but I wonder how many readers want to do that kind of research, and for the ones that do, I imagine some don’t want to share wha they find. I’m interested in what you have to say MeiHua, so wherever you wind up sharing, I’ll subscribe. I’d like to see this thread continue, so I’d hope for it to be here, but obviously it’s not my thread.

Hehe, yeah, mee tooo :slight_smile:

Since the last time, I have tried to make my own seasonal analysis, check this out(almost guaranteed not accurate though:/ =

The green line is data for 2013, big purple one is total mean, blue one is the mean for the last 5 years, red is the mean for the first 5 years of the study…

Havent tested many TA ideas yet, still trying to grasp R :wink:

It would be nice to have all the stat related posts located here in one central location. It would beat jumping from thread to thread IMO. I will post a few things that are candle pattern related in the near future. There may be very few interested but that will be their problem.

Why do these types of threads die? For a number of reasons. I happen to like knowing that my chances when doing X are greater than my chances when doing Y or that doing Z is a statistics-declared disaster so I should avoid it. Some don’t care. Also, many if not most methods in trading are quite subjective. It is hard to quantify subjectivity.

Anyway, I am all for giving a bit of CPR to the thread.

No I am very interested in seeing the research you and everyone else has done. I made the thread specifically for people from all possible trading approaches to share their understanding of the markets with the evidence that supports their opinions. Post away I enjoyed reading what you shared and i’m sure others did too. Contributing is not hijacking this thread.

This type of thread will never be popular, there is no leadership or structure in place to follow, no plan to make the readers rich, just a place to share what you find and the evidence that supports your claims and see if you get useful/useless feedback.

While I started the thread I have little to contribute as the field is not something I am experienced in, but I find that sharing ideas about real evidence is a great way to learn and or demonstrate ideas.

What exactly are you calculating? its not really clear. How are you isolating the seasonal cyclical factor and then parsing the data to calculate? why over the last 5 years first 5 years and total mean? are we looking for drift? considering that the euro is not say like corn, crude or other commodities which are locked in through either natural or fundamental cycles. i think it would be hard to resolve a drift with fundamentals. Though I am not certain your data speaks to that. Also how many years and what data feed was using in your study?

Hi, sorry for the unclear stats, I was just trying to produce my own seasonal tendencies… The data is for the 2001 - 2011. And the close price is transformed with rate of change…