Also for higher time frames according to your system you might need higher stopploss levels (above 50pips) which would be fine for standard and instituitional accounts and not for mini accounts with 2000$.
The best as far I have read from experts and from my experience is make decisions on a higher frame and execute on a lower frame. In your case I would say make decisions on 15min and execute on 5min.
Best Wishes,
Arun
Arun,
Just to make sure we understand each other, have a look at the picture. It is dJMA10 with threshold applied to filter out the noise. It has not been optimized whatsoever. It is just to give you an idea of what I mean by trading signal. Green means buy, red means sell and nothing means do nothing. Now this indicator is drawn shifted 2 bars left. If it was doing this realtime, I would not be here looking for ideas !!! This is what I would use as the desired output of the NN for the training.
What you have presented as trading signal is also correct but I believe that classifying the deisred output may lead to better performance of the NN. The simpler the task, the better it usually gets. Predicting an exact value of EMA is harder for the NN. Also, if PNN is to be tried in the future, it will be easier to implement this way and PNN are pretty good at classification. I am stubborn about this detail but it is just for the sake of NN performance.
For my last idea, I must admit it is hard to understand and to explain. I think you understood most of it but the trick is to use shifted indicator to do the trading system. For example, let’s use a simple MA crossover that are fantastic in trends but gets you profit back and bankrupt your account quickly in ranging markets. The idea is to use shifted fast and slow EMA and backtest it. You then fine tune the fast and slow periods and then create 2 NNs to predict the future values of the two EMA and trade the system using the forecasted values. Now my idea to use a NN in the backtests is in complex trading system. For example, if someone uses RSI, STOCH, MA crossover and DeMarker to enter their trades, why not just put all of them in a NN and train the NN to make profit out of these indicators. The trick is again to use shifted indicator values and when everything is optimized, you create NNs that will predict the values of all the indicators you are using in your trading system. The predicted values will be entered in the same NN that was used for backtest. This is why you end up with many NNs. This knid of approach is quite time consuming…
The idea is to not shoot in the dark. It is better to first identify a system that gives you good trading signals with shifted indicators and then predict these indicators. In the picture I attached, the signal would be quite good. The indicator catches most of the significant moves. It will also show bad signals from time to time but with good MM, the losses can be minimized.
Thanks again for your thread !
Arun,
Seems everybody is posting at the same time !!
The dBB is not needed as it does not give more information we don’t already have. Initial value would be around 15. The deviation is irrelevant if you take upper band minus the lower band. If you want to see what upper minus lower band gives, try the Waddah Explosion indicator. You can find it on Metaquotes web site. He uses upper minus lower bands for his orange-brown line. It shows quite well the volatility but he used a period of 20. 15 would be better as it reacts quicker. I would also consider JATR for volatility. It is quite fast and smooth. Standard ATR is too laggy.
The dRSI is quite efficient at pointing reversals with less lag. When used alone it gives a lot of bad signals but in conjunction with the other indicators I think it will add to the NN performance.
Finally, instead of using EMA5 from 1hour, you could use a longer period EMA on M15 as it would be easier to backtest unless your backtester can do multiple time frames. MT4 does not support MTF backtest so better use the longer M15 EMA like EMA20 or EMA34.
I misunderstood dJMA as a better filtered signal compared to dEMA when you mentioned it as a output signal but did not think of classification. Also my intention for not taking JMA was to avoid dependencies in the initial implementation. I agree to your argument that classifying the desired output might lead to better performance.
I think it would be easier first to forecast dJMA10 as you mentioned and later by analyzing at the output of the NN results we then can decide the thresholds and retrain the network with only output signals as BUY, SELL, FREEZE. What do you say:confused:
Back testing MTF in metatrader is not possible but as far as I know getting an indicator value of on a different time frame is possible. Infact in NeuroTrend I have used EMA of 1hr chart although the target was for 15min chart. The right trick I thought was to get the value of indicator based on time and not on Bar shift. I need to check this once again.
I will update the doc when I am back from work.
Thanks,
Arun
I got some PNN code from Russian forum. There are lot of works needs to be done but it’s a good place to start.
Arun, i really appreciated your hard work. I am sorry if i put other NN to this thread. If you think we need to start other thread for this PNN, please let me know.
Thanks for the code. I will look into it afterwards when I do some literature review on PNN posted by pegasus200. We plan to implement PNN later in the development process. I don’t mind posting the code in this thread as long as it is related to scope of our discussion.
I should thank you guys for your support and feedback. I learn a lot from these discussions and I envision that we will come up with a good system.
Arun,
The methodology I am thinking is to first identify a system that works good and then try to predict it’s indicator. Let me explain again.
I would use the JMA10 indicator and tune the threshold and shift values to get accurate signal (like the picture I attached in my previous post) before any training is done. Once you are satisfied with the desired indicator, you use these values as the desired output for training. There is no need to predict the JMA10 values and then tune the threshold and retrain, as you add extra work to find out the desired signal, which you can easily get using the original indicator !! The network will only be trained for giving signals.
It will probably imply that you write a file with the desired output values using a script or in an indicator code because of the shift (you can use the time instead of shift). You would then use this file when training, assuring that your training start time is synchronized with the start of the desired output vector.
I don’t know how your training software works but I believe that you need to write some files for the input and desired output values and then push them in the software. I will try to have a look at the software and give you a hand on the training part.
Regards
Hi, I did understood your methodology, but I was not sure if we should carry on first optimizing the threshold of the indicator and prefdict it or predict the indicator first and optimize it to retrain. Lets optimize the thresholds of the indicator first as you said and proceed.
Could you send me the dJMA implementation that was in the picture you posted previously. I will play with the thresholds and post them to collect the feedback.
I do already have code to output the required signals to the format of the software. I only need to implement the indicator signals we have decided to get my training and validation files. It would realy make things easier if you get a grab on the software as I would surely need your analysis on the results. Also we can be confident on the results by cross-checking.
Before that let us decide thresholds and tune the dJMA indicator as per our requirements.
Arun,
Here is the attached dJMA. I added the feature to have output values stored in only 1 buffer so it might be easier for you to extract the desired output vector.
The usual parameters of dJMA are Length, Phase and Shift. The applied price is from the PriceSeries.mqh. The threshold does not have hysteresis but we should be fine without it for now.The showsignal var is the saturation to +1 and -1 for dJMA values above the threshold when true. The usecolor will put the values into 1 or 2 buffers for display. When false, all the values are in the first buffer.
Finally, predicting the dJMA and then setting the threshold will lead to an accumulation of errors because the threshold is set to the predicted value instead of the more accurate shifted indicator. Anyway, let’s start with something and we’ll see. By the way, if you happened to follow the bogie NN v8 EA, you will see that it is a copy of the Cortex example back in 2003 (cortex.snowcrown.com) and when the EUR/USD will start to plundge, so will the bogie NN v8 EA (it is not doing well on downtrend unless you change the threshold points which can take time before you indentify the downtrend). This is one example that shows well what happens if you apply a NN to an indicator that is not well suited for signal when used alone. Anyway, I recommend you read the whole page on Cortex about the trading system(Neural Networks and Stock / Forex Trading).
Do you use the Java version of SNNS or you have and x-win emulator or run linux ?? I am on Winblows (oops, did I mistype something?) and don’t have any virtual machine installed nor x-win. I installed the Java version and it seems to be working. Have to go through the manual though… Will do my best to help you out with the NN training. I am quite busy with home renovations right now !!
Hi Pegasus200,
As I was thinking on the output signal brain neurons identified a method which they memorized long back… heheehe. Could you look at this presentation (attached below) by John Ehlers on creating a trading system based on non-linear filters. According to this presentation he adds percentage of ATR to the MA (in our case JMA).
The main idea is to…
Add & substract a fraction of ATR to the non-linear filter to create a hysteresis channel.
This fraction could vary for uptrend and downtrend.
Use this hysteresis channel as the threshold for the current bar MA to make decisions.
In our case this hysteresis channel would be
[B]JMA(t-1)+JATR/frac for +ve threshold
JMA(t-1)-JATR/frac fpr -ve threshold[/B]
frac is a real number which could be different for uptrend and down trrend or could be also the same.
Main advantages of this method are:
Simply we can say
[B]BUY = dJMA > JATR/frac
SELL = dJMA < JATR/frac[/B]
DO nothing when in between the hysteresis channel.
Actually the channel is similar to kettlers channel.
What do you think about this rather than blind detection of thresholds. Ofcourse here too we require blind detection of fraction but it adapt dynamically to the price action.
Best Wishes,
Arun
Arun,
This is a good idea. I have spent some times on channels (Bands, Keltner, High and Low MAs and JMAs, etc). The thing is that channels add lag to a signal. It filters out some bad entries but it also cut the profitable ones too.
Sometimes you just get out out the channel (which gives you a signal) too late like at the end of a consolidation period. I’d try it for shure but my suggestion is to use a narrow channel. I prefer to have an earlier “not so good” signal with narrow channel and use good MM like moving to break even after some pips are banked or scaling the position out to limit losses.
Another idea is to use longer period for the dJMA. It will filter out more noise. The entry will be lagging a bit but there will be less false entries. The stop loss would also need to be higher for compensation of larger market moves.
I also tried to add the volume like the force index indicator but volumes are not the same from broker to broker… This is unregulated market so volumes are often better left off…
This is easy to implement anyway so it is another thing to try.
I’ve been monitoring this post and read some blurbs about neural networks. I get the broad concept, but I’m mostly curious to know exactly how you go about training the network using MT4 data. I’ve looked at SNNS and JavaNNS, but I’m still lacking progress in uniting your suggested Forex data with a neural network.
Basically, I’m interested in knowing what data to extract from MT4 and how to proceed; should I be using the input indicators mentioned in your first post and if so, are they manually extracted or is there an automated way of doing this? Should I be collecting the data off the 15 minute timeframe?
Forgive my ignorance, but I’ve never studied neural networks before. Is training a Forex neural network beyond someone with limited experience?
I’d be interested in training other currency pairs on my own to see the results.
Thanks for your help
maybe should think adding non trending indicator such as chakins monetary flow.
because RSI, stochastic are trend following indicators
Nabil,
Good idea for the non trending indicator. The only thing is that this particular indicator is based on volumes. You will have different values from broker to broker (forex is unregulated and volumes are broker based) so the network would have to be optimized for each and every broker… Could be a bit complicated…
Voodoo,
NN are not an easy task but software like SNNS makes it much simpler for anybody to try. You need to have a good math background and you have to be patient and stubborn. It is hard to figure at the start but patience will get you there. Also, you need a good computer knowledge and some level of programmation because you generally have to write text files with you data to feed the NN.
I have not coded with SNNS yet but Arun did. He may be willing to help you but I suggest you to read the SNNS manual and also try the examples to know the data format it uses. This thread is more for the system developpement and is not a NN tutorial. I just want us to keep focused.
Basically, NN needs a vector of inputs and a reference vector (the output you want it to learn) for the training period. Once trained, you only feed it with the defined inputs and get the output. One important thing to know is that you input vector has to be manipulated to ideally fit within the range of your activation function. This is called data preprocessing. Popular activation function have ranges form -1 to 1 or 0 to 1. If you input the values of a moving average of USD/JPY of 110, then you will not be going very far since your weight will be very small and you will have floating point error accumulation in the network calculation. It would be better to take the 110 value and divide it by 100 before sending it to the network. This way you have a lower value and you weight will be in a decent range value. You have to be careful because if the 110 value jumps to 200, then you will probably be in the saturated region of the activation function and the network will not perform well. This is why we use derivative of the moving average instead of the moving average itself. The range is thighter and a saturation has less effect. You have to chose your inputs carefully. Also, too many inputs will need a great load of data to train the NN. This has to be carefully examined when creating the network.
The time frame is up to you. It depends what your trading style is. Short, medium or long term will guide you in choosing the time frame.
Keep reading on the web different tutorials and you will get there. There is no easy way out but it is achievable.
Hope this helps
P.S.: If anyone wants to learn NN or discuss about NN basics, please start a new thread in the beginner section so this thread will keep the momentum in the system developpement.
Hi Pegasus200,
I use JNNS right now as I have also windows on my machine and I did not get an alternative to the demo X windows server.
I should make clear here that I use tanh as activation function. There are two reasons for the choice 1) Training speed 2) since we use derivative it would be easier to fit the signals to ideal range.
Also important thing… the smallest unit for the derivative in the case of EURUSD will be 0.0001 which means if we use the following formula
signal = derivative(EMA, JMA, etc…) / point
then derivatives greater than 0.0001 will be mapped to ‘1’. I think such a fine resolution is not required so in NeuroTrend if you check the code I used the following formula.
signal = derivative(EMA, JMA, etc…) / point*10
which means derivatives greater than 0,001 will be mapped to 1. What is your opinion on this.
Also as you mentioned, from now onwards, we should let the ideas flowing on system development rather than discussing basics. We discuss theory on context basis.
Please verify the updated specification of the NN design. Excluded StochSignal as it is smoothed StochMain and does’nt say anything new. Word doc also attached.
Voodoo,
I suggest yoy to follow this thread as sooner or later we will be covering all the topics you are curious to know about, but its also a good idea to start a separate thread only for learning neural networks basics.
You need to write the signals from mql to text files according to the pattern file format of SNNS. text files are like an interface between mql indicators and SNNS. With the code in the first thread you can read the output of SNNS, create a network and initialize the network with weights and biases. You need a kind of script to write out your indicator values to a text file that is in SNNS pattern format. You can easily code it if you have basic knowledge of file handling in mql.
@ All,
Today, I could not get time to start with the dJMA analysis. Hopefully We can finalize the signals by end of this week. Mean while please verify if we can add or remove some indicators. Nabil I agree with pegasus that we should avoid volumes in the first phase due to their variation from broker to broker.
Best Wishes,
Arun
i know you guys talked about multiple moving averages
checks website help understand better what do with it.
Convergence and Divergence
The most important signals are taken from the spacing between the MAs in each group; not from crossovers.
* When moving averages within a group are parallel and close together, the group are largely in agreement;
* When the moving averages widen, this signals divergent views within the group;
* When moving averages converge, this is a sign that the group view is changing.
Trend Strength
* Parallel long-term MAs signal long-term investor support and a strong trend; and
* Short-term MAs tend to bounce off the long-term MA group.
Trend Weakness
* Both groups of MAs converge and fluctuate more than usual.
Trend Start
* A change in price direction accompanied by expanding MAs in both groups.
Short-Term Reversals
* The short-term group cross over, diverge and then again converge; while the long-term group remain parallel.
alot of people use it for scalping to help determine things mention above
and maybe you should try that.
Hi Nabill,
MMA sounds interesting as the analysis is based on divergence rather than cross overs. Do you know any indicator that is a squeezed version of MMA. Including so many MA’s I think would increase learning curve.
Cheers!,
Arun
to learn more about it
rainbow 3d moving averages in google click on the first one incredible charts. they start remove my links.
my friend have seen indicator but doesnt work well but moving averages are suppose show different things. if you have muliple moving averages then neurual network will learn those mma relation to price movement.