Hi Proximus,
The answer to your problem seems simple. For one, and I say this with the utmost respect, just build the damn thing and stop talking about or involving yourself in dialogue with people that can’t help you to do just that. Sure they have useful opinions, but in the end, none of them really give you anything you can use to go about your potential project. I’ll try to help with a few points, hopefully, being a programmer yourself, you can extrapolate these points and modify them to suite a custom approach to analyzing large amounts of data.
[B][U]BASIC PREMISE OF A NEURAL NETWORK[/U][/B]
I should say that the information I wish to divulge relates to mainly backward propagation neural networks. That simply means that the networks send whatever error was inherent in their calculations to previous neurons in the network for recalculation, more on this later.
The structure of an artificial neuron consists of two units: a summation unit and a unit with a transfer function. A neural network is simply an interconnection of these artificial neurons arranged in layers. There are three primary layers: the input layer, the hidden layer, which is subdivided into a number of secondary layers and finally, the output layer. The connection between layers is as follows:
Input layer->Hidden layer [sub-layers]->output layer.
Each layer has a certain number of neurons. For example, a simple network can have 3 neurons at the input level, 5 neurons at the hidden layers, and three neurons at the output layer. There are also different types of connections, either full or sparse. For instance, in the simple neural network cited, each of the 3 neurons in the input level could be connected in all the five neurons at the hidden layer to form a full connection. These five neurons at the hidden layer could further be connected to another five neurons in a sub-layer, which would finally be connected to all the three neurons in the output layer to form a complete fully connected network.
The connections from one neuron in one layer to another neuron in another layer have bias weights. The neuron itself also has a bias value that is set to a random number once the neural network is initialized. This explains the summation unit within the neuron. It basically sums all the connection weights from the incoming data, adds its own bias and finally sends it to the transfer unit, which would result in the final output of the neuron. Before I write the formula for summing up data within the summation unit and then taking it through the transfer function, please consider the simple network below: [sorry, don’t have much time to upload images]
N1------N3*
* * *
* N5****
* * *
N2------N4*
In the above ‘diagram’ you can see that neurons N1 and N2 form the input layer. Neurons N3 and N4 form a hidden layer with no sub-layers and finally, neuron N5 forms the output layer. That’s a basic feed-forward network, it ‘feeds’ anterior neurons in anterior layers data. A summation unit works as such:
Dim netValue As Single=bias
For Each Input_Neuron connected to This_Neuron
netValue=netValue+(Weight_Associated_With_Input_Neuron*Output_of_Input_Neuron)
Next
To put it in a different way, using N3 as a reference point:
Net Value of N3=N3.bias+(N1.OutputWeight of connection from N1 to N3)+(N2.OutputWeight of connection from N2 to N3).
Do the same for all the neurons in the hidden layers, or receiving data from other neurons. That’s just how the summation unit of a BPN works. Now to the transfer unit, that modifies the Net Value calculated:
Output of Neuron=1/(1+Exp(-NetValue))
As you can see, it’s a sigmoid transfer function. And that’s it really. The next thing is to get the error of the network and use it to modify the bias of each neuron.
–GETTING THE ERROR/DELTA–
The delta or error is simply the difference between the desired or actual output and the calculated output. The steps involved in calculating the delta of each neuron in all the layers are as follows:
- Get the error of each neuron in the output layer.
- The error thus obtained is used to get the error in the previous layer.
- This error is then used to get the error of the previous layer and so on until we reach the first layer.
As you can see, it propagates the error from the output layer backwards [hence its name, the Back -Propagation-Neural-Network], using it to modify the bias value of each neuron, before the feed-forward process is re-initiated. The general equation for getting the delta of a neuron is:
Neuron.Delta=Neuron.Output*(1-Neuron.Output)*Error_Factor
The Error_Factor of the neurons in the output layer can be calculated directly, because we already know the expected output of each neuron in the output layer:
Error_Factor of An Output Layer Neuron=Expected_Output-Neuron’s Actual Output
The error factor calculation for a neuron in a hidden layer is somewhat different, to calculate it:
- The delta of each neuron to which this neuron is connected is multiplied with the weight of this connection.
- These products are summed up together to get the error factor of a hidden layer neuron.
In a nutshell, a neuron in a hidden layer is using all the delta of all connected neurons in the next layer, along with the corresponding connection weights, to get the error factor. This is, as can be obviously inferred, due to the lack of direct parameters that can be used to calculate the error of neurons in the hidden layer as we did in the output layer.
So, after getting all the errors of all the neurons in all the layers, correction of both weights and bias with respect to the error or delta is performed, to get more accurate output for the next training/feed-forward cycle. Connection weights and bias are both free parameters. The neuron should update all the forward weights associated with it.
The Pseudo-code to do all this [updating weights and biases (free parameters) in all layers]:
Update free parameters of all hidden-layer neurons
For each layer in HiddenLayers
For each neuron in layer.Neurons
neuron.UpdateFreeParams()
Next
Next
Update free parameters of all neurons in output layer
For each neuron in OutputLayer.Neurons
neuron.UpdateFreeParams()
Next
Anyways, that’s how I’d go about it, you can make up your own way of doing this. Finding the bias value of a neuron is rather easy, if the Learning Rate (I’ll explain what learning rate is later) is constant:
New Bias Value=Old Bias Value+Learning Rate1Delta
The new weight associated with a neuron can be gotten by:
New Weight= Old Weight+ Learning Rate1Output of Input Neuron*Delta
And that’s all. I should say that I’ve left a lot of theoretical information, I wanted to be as direct as I could and also to show using actual formulae and pseudo-code how this could be done. Oh, the learning rate is just that, how fast the delta approaches a certain minimum. In mathematical terms, the gradient vector of the error surface should be gotten. Usually, the step size is proportional to the slope (so that the backward-error-propagation algorithm settles down in a minimum) and to the “special” constant: the learning rate. You should know that the correct setting for the learning rate is application-dependent and is typically gotten via experiment; it could also vary with time, getting smaller as the algorithm ensues.
I don’t know if I just wasted 25 minutes trying to explain this or if I’ve actually managed to tell you anything useful. As a side note, I suggest that before you begin building this thing, or before you try to implement any algorithm that attempts to model the function that associates a certain input to a certain output, you read up on Information theory, and the various characteristics of stochastic data. Because the data obtained from the forex market is random, various patterns within small time frames can be ‘learned’ and these patterns will never change, provided the algorithm ‘changes’ with its market-presence. Also, any efficient market hypothesis loses its value once it becomes public. That means that any ‘Holy-Grail’ that is mass-accepted will eventually fail, due to just that, everybody’s using it. However an algorithm that can potentially withstand this and adapt to any ‘market’ could be called, in my opinion the ‘Holy-Grail’. Traditional Neural Networks, not just BPNN, but any multi-layered perception network from radial basis function networks to generalised regression neural networks will not work in the long run, unless the function that has been modeled by the hidden layers, changes with time, where time is the desired trading time frame, i.e. a second, a minute, an hour, a day etc. How to determine this change with time, you can think about yourself. Lastly, what inputs to connect with what outputs also becomes an issue. Should you use the day’s high as input and the day’s low as output? Should you use the price at time t/2 as input and the price at time t*4 as output? What is the desired inference? That certain prices lead to other prices? That there is a connection between the time of price x and the time of price y? Without certain constant assertions, it becomes rather easy to develop a misconstrued attitude towards random data. But, with various mathematical proofs within Information theory that deal with the nature of random data, you can, by extrapolating these proofs to the forex market (again, because the data is random) develop a system that will always yield consistent results. Then again, you could just ignore all this and use brute-force methods, you could get a program called Pythia Neural Networks (it’s free, google it), use certain inputs with certain ouputs over a long time period, say 10 years, and just see what happens when you use input i with output o. Well, that’s all I have to say, I hope I’ve helped, you can ask where I have not be clear.
