Probability of Future Market Conditions

Trading Discussion
1

Greetings folks,

I have been doing a little research and I think I have stumbled across something that is probably as close as you will get to a Holy Grail.

I think irrespective of the trading strategy you use, one of the biggest problems facing traders is the inability to predict the future conditions of the market features that you rely on to profitably trade your system. Time and time again, trade entry signals, break out of formation and return as false flags.

If we were to agree that the model for profitable trading is:

  1. Finding a system suitable for you
  2. Finding the market conditions in which that system is profitable
  3. Finding a way to predict if those conditions will exist in the near term (or will continue to exist for the duration of your open order)

Then there seems to be a reliable way to tackle the third challenge that enables you to ascribe a probability value to the future condition or behaviour of the market feature(s) that you rely on to be profitable with your system.

For example, if I trend trade with a simple price cross of the SMMA(14) + One Sigma, I would be interested in (1) the price continuing in the direction of the trend (feature 1: price movement) and (2) the True Range rising (Feature 2: increasing volatility) as the conditions that are most likely to ensure a profitable outcome.

One way to find the probability value for those conditions existing in the near term (in the incoming few candles), would be to look at the state of the auto correlation on that feature and use the transition probabilities from that state to make an inference on the future condition of the feature.

The theory is very simple, but depends on some knowledge of:

[B]Auto Correlations[/B]:

Autocorrelation || Time Series Introduction

[B]Markov Chains (to model autocorrelation states):[/B]

[B]Transition Probability Matrices:[/B]

[B]Conditional Probabilities:[/B]

2

As an example, I want to look at the price movement as a market feature for a simple trade in the direction of the trend on GBPUSD H4.

I will use 2010 - 2014 (end) to create a model which will be tested on EURUSD 2010 - 2014 (end)

(1) Capture the Price movement on each candle (Close - Open)
(2) Copy and paste the price data in a column in Excel.
(3) Create a copy of those prices in an adjacent column (Column 2), removing the first value, so that the values in the Column 2 are identical to Column one except being one step behind.

(4) use Excel to find the correlations (in this case a serial or auto correlation) - I am using the correlation in the last 7 bars (7 seems to be a sweet spot)

(5) Create a new sequence of these correlations down your data set Column 3. The correlation values are normally distributed but the average value might not be exactly 0, usually there is some skew.

(6) The correlation values range from -1 to 1. So dividing that interval into 20 to create 20 separate states where:
-1 <= Correlation Value < -0.9 is [B]State 1[/B] and -0.9 <= Correlation Value < -0.8 is [B]State 2[/B] etc…

(7) Going down the list of correlation values in Column 3, label the state that each correlation value is in (Column 4)

(8) Create a 20 by 20 matrix Matrix One

(9) Going down the correlation states in Column 4 from top to bottom, plot the transition frequencies between states. So for example if the first value in Column 4 is 13 (State 13) and the second value is 15 (State 15) then from left to right in Matrix One:

I find State 13 in the vertical axis
I find State 15 in the horizontal axis
I increase the call in which they cross by 1

(10) Doing that for all states in column 4, I should now have a transition frequency matrix (Matrix One)

(11) if I sum all the values across the rows of Matrix One, I know how many times the correlation started in that state and I can use that with the transition frequency to other states and or itself to create a transition probability matrix; (Matrix 2)

3

Picture1.jpg700×312 63.2 KB

Transition Probability Matrix for GBPUSD H4 from Jan 2010 to Dec 2014. Correlation States from last 7 Bars.

Altogether circa 8000 entries.

4

You can see that if the correlation is in State 1 or 2 (Vertical) then the probability that the next sequence of correlations on 7 candles will increase, is very high.

[B]State 1[/B]: Probability next 7 in higher state = (1 - (0.3)) x 100 = 70%
[B]State 2[/B]: Probability next 7 in higher state = (1 - (0.0448) + (0.2239)) x 100 = 73.13%

Using the transition probabilities as a predictor is straight forward.

If I consider the sequence (1,-1,1,-1,1,-1,1,-1) as a showing how the last 8 bars closed (up = 1, down = -1).

Then I can create an duplicate sequence lag 1 bar to look at the auto correlation as so:

Correlation between

[ul]
[li]Original Sequence (1st 7 Bars): (1,-1,1,-1,1,-1,1)
[/li][li] Copy Sequence (Last 7 Bars): (-1,1,-1,1,-1,1,-1)
[/li][/ul]

The ‘auto correlation’ in this case is -1 (State 1). If we introduce a 9th Bar with Value -1 and now look at the auto correlation in the last 7 Bars.

Correlation between

[ul]
[li] (-1,1,-1,1,-1,1,-1)
[/li][li] (1,-1,1,-1,1,-1,-1)
[/li][/ul]

The Correlation value is -0.75. So the correlation value has increased from -1 to -0.75 due to the introduction of the 9th bar which was in the same direction as the 8th bar. In the previous case, all the bars in successive time were in opposite directions, with the introduction of the 9th bar, this time in the same direction as the 8th bar, the correlation in the last 7 has increased.

So going back to the transition matrix, we know that if the correlation state in the last 7 bars is very low ( 1 or 2) there is a circa 70% probability that the incoming bar will increase the correlation value (therefore correlation state) which can only happen if the incoming bar is in the same direction as the last bar observed.

i.e. there is a 70% chance that the 9th bar will be in the same direction as the 8th bar given that the correlation of the previous 8 bars was in state 1 (Correlation value -1).

The general principle is that if you observe the auto correlations of any market feature, you can calculate the transition probabilities between the auto correlation states. If the auto correlation is in a very low state, then there is a very high probability that the incoming bars will have a higher auto correlation, that implies that the incoming bar is more likely than at any other time to have similar characteristics to the bar just closed.

So if the market feature being looked at was the True Range. If the true range in the last bar is above the ATR for a period and rising, where the auto correlation of true range is in a very low state, then it is very likely that the incoming and future bars will also have a higher and rising True Range.

5

I think it would be much better to look at the transitions in correlation state in the previous few candles (flight path) to construct a conditional probability of the correlation state increasing or decreasing at any given time. I’m sure there are better ways to optimise the probability of an outcome than simply looking at the current correlation state.

6

Going to do a simple test on EURUSD for the period Jan 2010 to Dec 2014. I will test 2 different trading strategies and only look at the market feature price movement.

The First strategy is a simple directional trade, the second strategy is a trend trading one, both with and without looking at the correlation states of the market feature; price movement.

[B]Trade One[/B]

[ul]
[li]On the entry of each new H4 Bar, if there are no live orders enter a trade.
[/li][li]If Price > SMMA(14) Buy, if Price < SMMA(14) Sell.
[/li][li]R:R Ratio 1:2.5
[/li][li]Risk = £1,000 per trade on a 100k account
[/li][li]Stop Loss = 40% of last True Range
[/li][/ul]
[B]Trade Two[/B]
[ul]
[li]Simple Trend Trade, Buy only
[/li][li]If the Price Crosses SMMA(14) then goes above SMMA(14) + Sigma One then enter in direction of trend
[/li][li]Close if price Crosses SMMA(14) again in the opposite direction
[/li][li]Risk = £1,000 per trade on a 100k account
[/li][li]Stop Loss = SMMA - Sigma
[/li][/ul]

7

Test for trade one produced the following results.

(1) No discriminants - enter trade if no open orders exist.

EURUSD - Directional No Correlation.jpg700×171 25.5 KB

Win Rate = 29.69%

(2) Discriminate based on the Price Auto Correlation: Configuration - if last 3 observations of 7 bars all had a correlation value below -0.8 and there are no open orders, then enter. In this case I hope the conditional probability of the correlation state increasing given the previous revolution of the correlation values in low states will produce even better results.

EURUSD - Directional Trade.jpg700×171 19.7 KB

Win Rate = 42.50%

8

This looks incredibly complicated at first sight, however you’ve clearly put in a lot of work here ropunzel.

I look forward to following, or rather trying to keep up at this moment in time.

It’s fair to say that this is the first time I’ve seen a thread which really does go down the rabbit hole of pure statistics, kudos deserved!

9

Test for Trade Two (Buy Only)

(1) No Discriminants

EURUSD Trend Trade.jpg700×171 21.9 KB

[B]
Average Profit Trade = £1,254.20
Average Loss Trade = £-765.56

Implied Win Rate to Break Even = 38%
Actual Win Rate = 33%

Relative Drawdown = 49.48%
[/B]

(2) Discriminate by Price Auto Correlation: Last 2 auto correlation values should be less than -0.8 (high probability that the price will continue to move in the direction of the trend)

EURUSD Trend Corr.jpg700×171 17.8 KB

[B]
Average Profit Trade = £2,599.43
Average Loss Trade = £-437.37

Implied Win Rate to Break Even = 14%
Actual Win Rate = 48%

Relative Drawdown = 4.65%
[/B]

10

Greetings, thanks for the post. I think I will have to look at the posts again and find better way to describe what I am trying to do.

11

I am going to test to see if you can find edge with the auto correlations on trades entered at random. At it’s most optimal, randomly generated trades should produce a 50:50 split between buys and sells and also alternate in direction on each turn, so instead of generating entries at random I will fast forward to generate alternating trades for perfect random behaviour.

There are market conditions in which even random trades can have edge. The sharp fall of GBPUSD on Brexit would be the perfect illustration of those conditions i.e.

[ul]
[li]When the market is Direct - most of the price movement or TR is in one direction from the open price versus the other &
[/li][li]When the TR is high and rising - the number of pips moved is high so multiple trades take place in the candle
[/li][/ul]
If we take the values High - Open (H-O) and Open - Low (O-L).

If (H-0) > (O-L) then Directness = (H-O)/(H-L) else Directness = (O-L)/(H-L), possible Directness values range from 0.5 to 1


Where the Long Legged Doji and Four Price Doji represent Directness values on or around 0.5 and the Dragonfly Doji and Gravestone Doji represent Directness values of 1.

So if you imagine trading perfectly random entries (alternating) in a Direct market with a high TR, for example GBPUSD on Brexit with an R:R higher than 1:1, say 1:2.5. As the price moves steadily down, every other one of my alternating trade wins (in the Brexit case the sell trades) giving me a win rate approaching 50% whereas I only need a win rate higher than circa 29% to be profitable because my R:R is 1:2.5.

So I can possibly use the auto correlation states to find probability values for the likely future condition near term of the two features that can bring Edge to a random entry trading system:

[ul]
[li]Directness &
[/li][li]A high and rising True Range
[/li][/ul]

What I am going to do instead is enter random trades with lotsize 0.01 and track the performance of my system by recording the sequence of wins and losses. I will then find the auto correlation states of the last 7 trade outcomes and use that to predict if the system is likely to increase or decrease in performance (be profitable in the next few future trades). That way I will be indirectly predicting Directness and True Range.

12

Test. Enter alternating trades on EURUSD lotsize 0.01 with stop Loss = 10 pips and R:R 1:2.5. Looking at the transition probability matrix for the auto correlation states on GBPUSD in the same period, I can infer that if the auto correlation state is 1 i.e. -1 <= auto correlation value < -0.9, then the probability that the current trend (performance) continues is 67%.

Therefore if the auto correlation state is 1 and the performance in increasing then increase my lotsize to give me £1000 risk on loss.

Result:

Random.jpg700×171 24.1 KB

[B]The win rate for all trades (0.01 lotsize and higher risk) = 26.73%

The win rate for just the higher risk trades = 33.33%[/B]

This tells me two things.

(1) The Principle Generalises Well - You should be able to apply the same principle to just about any trading strategy and on any asset class, although you would have to model each separately to custom fit the best conditional probabilities. So you could in theory run multiple strategies on multiple asset classes and in multiple time frames simultaneously. An over arching system could find probability values for the future conditions of the features you need to be profitable with each strategy and deploy the systems that are most appropriate for the moment.

(2) Risk Management Application - The random entry trades system was effectively a risk management application because I was predicting performance (profitability) as a feature of my system. You should therefore be able to apply the same principle to predict the performance of other trading systems to determine whether to scale up or scale down your risk.

With a probability value for the likely future performance (profitability), one could run a portfolio comprising different simultaneously active trading strategies, using the predictor to allocate funds to each strategy in a way that would maximise the overall risk adjusted return of the portfolio.

13

I think there are a number of possible applications, so I am going to stick to two for further research to avoid reading too widely.

(1) Immediate application of what I have so far to the most profitable trading system (Trend Trading)

(2) To write an Indicator that can find Edge in Price Action Trading on any instrument (See below)

I used to PA trade using a custom EA for position sizing; an EA with two horizontal lines on the chart, one line was for where the stop loss should be, the other line was for where the take profit should be (all from the market price)

So when you press a button on the EA (“Enter Trade” button), it would create a Buy or Sell order with the TP and SL in place of the horizontal lines

So if you consider the 2 horizontal lines as two absorbing barriers of a Stochastic process and you have the correlations (of price movement) in the last 7 bars, and the history of transitions between the correlation states in the last 5 bars (the previous trajectory through the correlation states), and the correlation state to state transitional probabilities.

You want to be able to work out the probability of hitting one of the horizontal lines before the other at any given point in time - So the probability of winning a trade with those SL & TP parameters at that time.

With that information and a very big if, if I can get that right. Then with the combination of those probabilities and the minimum win rate required to be profitable implied by the R:R on the position of the two horizontals lines to the Market Price.
You should be able to find where the SL & TP should be placed to find edge trading PA manually (or with a program) at any given time.

That is you could position the SL & TP from the market price to give an R:R of 1:3. You would therefore need a win rate above 25% (1/4) to be profitable long term, and the indicator suggests that the probability of winning that trade is 36% say. Then enter trade.

15

interesting read, thank u

16

Looks good. I think you could definitely do well with that if you are also allowed to modify the sheets with custom VBA codes.