Machine Learning Project development Step 3: Design of Artificial Neural Network (Supervised Learning Algorithm):

 

Machine learning project development: Design of ANN

Artificial Neural Network(ANN) is the most suitable architecture in many of the AI applications. The back propagation Feed Forward Neural Network is popular in solving many non-linear multivariate and complex problems.

 Design of ANN: Important steps in designing ANN algorithm is

 

1.   Collection of data 

2.     Structure of ANN

3.     Training of Network

4.     Testing of Network

5.     Validation  of Neural Network

6.     Performance Analysis

      1.Collection of data: To train the network strong database is required. There are many websites available that will provide requires data set for AI applications. Few  dataset finders are

Google Dataset Search,Kaggle,UCI Machine Learning Repository,VisualData,CMU libraries Big Bad NLP Database etc 

2.Structure of ANN: The first step in the Design of ANN is to decide the structure of neural network.

Structure of back propagation Neural Network consists of the input layer, Hidden Layers and Output Layers.

A number of hidden layers and nodes in that hidden layer are based on the complexity of data. In most cases, it is trial and error. Start with one hidden layer and approximate the number of nodes in that layer. Check the error and go on adding the number of hidden layers and nodes to improve the performance

3.Training of neural Network: Total data set is divided into training data set, Testing data set and validation data set.

4.Testing of Neural Network

5.Validation of Neural Network

Once the structure is decided algorithm is used to train, test and validate the network. In Matlab, various training algorithms are listed below. The  Levenberg Marquart Algorithm will be giving the best results in many of the applications.

 

Training Algorithms:Optimization Techniques:

1.     Steepest Descent

2.     Variable Learning Rate.

3.     Resiliant Back Propagation.

4.     Conjugate gradient

                                      I. Fletcher-Reeves Update

                                      II. Polak-Ribiere Update

                                      III. Powell-Beals Restarts

                                       IV. Scaled Conjugate Gradient

5.       Quasi-Newton Methods

I.  BFGS Algorithm (Broyden, Flertcher,                                     Goldfarb and Shanno)

                             II.  One Step Secant Algorithm

6.       Levenberg Marquart

•      newff(PR,[S1 S2...SNl],{TF1 TF2...TFNl},BTF,BLF,PF)

•      PR - R x 2 matrix of min and max values for R input elements.

•      Si - Size of ith layer, for Ni layers.

•      TFi - Transfer function of ith layer, default = 'tansig'.

•      BTF - Backprop network training function, default = 'traingdx'.

•      BLF - Backprop weight/bias learning function, default = 'learngdm'.

•      PF - Performance function, default = 'mse'.

 

The transfer functions Tfi such as tansig, logsig, or purelin.

The training function such as trainlm, trainbfg, trainrp, traingd, etc.

 trainlm is the default training function because it is very fast, but it requires a lot of memory to run.

 

 

Speed of training algorithm depends on

–     Complexity of the problem.

–     The number of data points in the training set.

–     The number of weights and biases in the network.

–     Error Goal.

Weights are updated in two different ways

–     Sequential Mode

–     Batch Mode