How to change the activation function in ANN model created using toolbox

I've created this model by editing the codes from the toolbox. The purpose of this model is to train the network with operating data from a turbine. the data is normalized and then the target will be set according to the actual fault occurrence which tagged as "1" and during normal operation "0". I will be comparing the result of several training function, the number of neuron, the number of layers, and activation function.

% This script assumes these variables are defined:
%   data - input data.
%   target - target data.
% load data
load data.mat;
load target.mat;
x = data;
t = target;
% Choose a Training Function
% For a list of all training functions type: help nntrain
% 'trainlm' is usually fastest.
% 'trainbr' takes longer but may be better for challenging problems.
% 'trainscg' uses less memory. NFTOOL falls back to this in low memory situations.
trainFcn = 'trainbr';  % Bayesian Regularization
% Create a Feedforward Network
hiddenLayerSize = 18;
net = feedforwardnet (hiddenLayerSize,trainFcn);
% Setup Division of Data for Training, Validation, Testing
RandStream.setGlobalStream(RandStream('mt19937ar','seed',1)); % to get constant result
net.divideFcn = 'divideblock'; % Divide targets into three sets using blocks of indices
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;
%TRAINING PARAMETERS
net.trainParam.show=50;  %# of ephocs in display
net.trainParam.lr=0.05;  %learning rate
net.trainParam.epochs=10000;  %max epochs
net.trainParam.goal=0.05^2;  %training goal
net.performFcn='mse';  %Name of a network performance function %type help nnperformance
% Train the Network
[net,tr] = train(net,x,t); 
% Test the Network
y = net(x);
e = gsubtract(t,y);
performance = perform(net,t,y)
% View the Network
view(net)

The questions are: Is it correct to use this code below and will it affect the function of my model?

RandStream.setGlobalStream(RandStream('mt19937ar','seed',1)); % to get constant result

How to add another hidden layer? How to change the activation function for each layer? What is the best plot to show the capability of the ANN model to detect the fault of the turbine earlier than existing control system. Please advise me if there are anything that can be corrected/improved.

[scadaprog]

Hi Rahmat,

Yes by forcing the pseudo random generator to use the same seed everytime the script runs will result the same weights in your network. You can add a layer manually and set all the parameters including the activation function like this:

net.numLayers = 3;
net.layerConnect(3,2) = 1;
net.outputConnect = [0 0 1];
net.layers{2}.size = 8;
net.layers{2}.transferFcn = 'tansig';
net.outputs{3}.range = [1 20];

or you can just create a new network: net = feedforwardnet ([18 8],trainFcn);

The best plot is of course dependent on your data and what characteristics of that data your network should learn. But for a start you can just plot your target on top of your network’s output:

plot(1:length(t), t, 1:length(y), y);

Also you can check the built in performance plots on the GUI.

[Rahmat Izaizi Ismail]

Thank you scadaprog, your answer really help me with my immediate problem Thank you also to Greg, your answer provided more depth and insight for me to further improve the model and my understanding. It is much appreciated. There is so much to be done to understand this further. I will use my weekend to analyze and adopt this information for my studies.

Answer 1

[Greg Heath]

You are approaching the problem in exactly the wrong way.

The multilayer perceptron with one hidden layer is a universal approximator. The only reason to use more than one hidden layer is to reduce the total number of unknown weights by reducing the total number of hidden nodes (i.e., H1+H2 < H).

Ntrn training pairs of I-dimensional inputs and O-dimensional output targets yields Ntrneq = Ntrn*O training equations. The best way to obtain a robust design that tends to be resistant to noise, interference, measurement and transcription error is to MINIMIZE the number of unknown weights that yield an acceptable solution. If possible, Nw << Ntrneq is desirable.

 1. Use FITNET (calls FEEDFORWARDNET) for regression and curve-fitting
 2. Use PATTERNNET (calls FEEDFORWARDNET) for classification and pattern-recognition
 3. You have a classification problem. Start with the simple code in
     help patternnet
     doc patternnet
 4. If there are c classes, the target matrix columns should be columns of eye(c): O = c.
 5. The relationship between trueclass indices 1:c and the target columns is
 target           = ind2vec(trueclassindices);
 trueclassindices = vec2ind(target);

 6. Before starting the design, get a "feel" for the data. This may include
  a. plot inputs
  b. plot targets
  c. plot targets vs inputs
  d. standardize inputs to zero mean and unit variance using zscore or mapstd.
  e. Repeat a and c 
  f. Remove or modify errors and outliers.

 7. Start simple with the example used in the help and doc documentation.
  help patternnet
  doc patternnet

 8. You only have to vary 2 things 
  a. Number of hidden nodes (want as small as feasible) 
  b. Initial random weights

 9. This can be accomplished with a double for loop as I have illustrated in zillions of examples in the NEWSGROUP and ANSWERS. Search results

 NEWSGROUP HITS
 greg patternnet Ntrials  8
 ANSWERS HITS
 greg patternnet Ntrials           60
 greg patternnet Ntrials  Hmax     22
 greg patternnet Ntrials  Hub      17
 greg patternnet Ntrials  Hub Hmax 10

Hope this helps.

Thank you for formally accepting my answer

Greg

[Khawaja Asim]

This was the kind of discussion I needed for my work. Thanks. But I didn't understand one thing. Why "targets" should be in the form of columns of identity matrix? Means If I have only two targets for my training data, say 1 & 0. I use a single target vector having either 1 or 0 corresponding to every training vector. Isn't it right?