Neural Network Predictive Modeling

Predictive Neural Network (NN) is a very powerful predictive modeling technique. Neural network is derived from animal nerve systems (e.g., human brains). The heart of the technique is neural network (or network for short). Neural networks can learn to perform variety of predictive tasks. For example, it can be trained to predict numerical values, say, future stock prices, probability, expected values, etc. In addition, it can be used in tasks that involve classification.

Neurons in Biological Neural Network

The core of neural network is (network-) nodes. Neural network nodes correspond to neurons of nerve systems. Anatomy of neurons (or neuro cells) is shown at the left figure. Nucleus contains DNA of neuro cells. Axons connect neurons and deliver neuro signals to other neurons. Dendrites receive signals passed through synapses. Synapse is the narrow gap between axon terminals and dendrites.

Neurons receive signals from other neurons and combine them. Combined signals are delivered to other neurons which may receive different level of signals as strength of connected axons will differ. The right figure illustrates this.

Artificial Neural Network

The main function of neurons is simple. That is, neurons combine input signals and pass to other neurons. When this simple function is organized into a network, it renders a very powerful system. Artificial (or computer) version of neurons are referred to as nodes. Artificial neural networks consist of layers of nodes and links between neighboring layers' nodes. The following figure is an example of artificial neural network. (You may consider it as a credit scoring or insurance scoring predictive model which predicts score for credit/insurance applicants.) The first layer is input layer. Nodes of input layer represent input fields or values of categorical input fields. In the figure, there are five input nodes: "Female", "Male", "Income", "Age", and "Education". The last layer is output layer. Nodes of output layer represent either prediction values or predicted class names. In the figure, there is a single node for "Score". The rest of layers are called hidden layers (or middle or internal layers). There may be zero or more hidden layers, normally a single hidden layer. The figure has three hidden nodes in a single hidden layer.

How Neural Network Predicts?
Links in the network represent axons of biological counterparts. As each axon of neurons reacts differently, each link is assigned with a different weight. Weights of links are the essence of neural networks. With weights, networks make prediction as follows. First, known values of input fields are presented to the nodes of input layer. Then, values are propagated towards the nodes of the output layer. In this process, values are multiplied with weights, summed and, then, applied to a non-linear function. Note that this is exactly how neurons combine input signals. Weights are designed in such a way that for given input patterns, values of the output layer reflect the values of actual outcome.

Neural Network Training

Weights of networks represented by links. Weights are computed by a method known as back-propagation. Computation of weights is called network training. Neural Networks are trained by "showing exemplary data and making corrections repeatedly", until networks fully learn patterns hidden inside data and are able to predict accurately. Network training is performed by repeating the following procedure. For each input training data record;

1. Present input data values to nodes of the input layer.
2. Propagate the presented input values towards the output layer (forward process).
3. Compare with the values of output nodes to the actual values of training data.
4. Correct the differences of outcome and propagate towards the input layer (backward process).

Normally, training data records are applied many times before network is actually used. Network training is a repetitive process. In the beginning, networks are trained coarsely. Then, they are refined by repeated application of input data. After networks reach a certain maturity level, they are used or deployed for value prediction.

Download of decision tree & neural network software is available for eavaluation from here.