how can i find the best ann architecture?

i need some help for building the best performance ann architecture for rainfall forecasting in a village in indonesia with 360 data (months in 30 years). First of all i am a beginner in ann also matlab, i just have a brief understanding of them, just a little bit. iam using nntool in matlab for processing (gui version), i dont even know how to get the syntax from the nntool gui in command window, why am i not using time series ann tools? because its to complicated, just need the simple net to process. Here is the problems and questions:

1.i have a problem for initializing or preparing the data, i.e iam using 2x120 input matrices with 2 inputs and 120 variables, and 1x120 for data target, i just want to predict the data in the next 1 year (it will be 12 data in months), all of the data is rainfall data. My understanding and logic of that kind of data construction is like this "if we have 360 data, and have 2 inputs variables, just divide the data into 3 so it can have same data samples for inputs and targets (cause i think that data target samples must be equal with the input samples), is that logic correct or wrong? just need to know how to prepare our data.

2.the training test of that architecture is really bad (using default training parameters) the mse is around 10^3-10^4, before that i have some questions, is it the performance value in training gui (nntraintool) at the progress table shows the real mse of ann performance? cause i just assume it like that, or if its wrong, how can i get the mse? I just need the mse, the %error and the predicted data for the final conclusion. what is probably the best training parameter for rainfall data (seasonal data)?enter image description here

3.next question is already mentioned in above, is it possible to shows or create the code or syntax from nntool gui? and if it yes how?

for the number of input's nodes, iam using the lagged correlation from the time series scatter plots of the data when it achieve the stasionary (the significant ACF and PACF) and the data divides for training, test and validation using random. iam sorry for my bad explanation (and also english) hope everyone could understand it. Thank you.

[Gede Sukrisnha]

here is the data 360 data in months for 30 years or just with this:

Answer 1

[Greg Heath]

 >why am i not using time series ann tools?
 because its to complicated,

Ridiculous.

1. The help and doc documentation will walk you through the basics.

 help narnet
 doc narnet

2. Other example datasets are available

 help nndatasets
 doc nndatasets

3. You may have to design multiple nets to mitigate bad sets of random initial weights.

4. I have posted many design examples in the NEWSGROUP and ANSWERS. For example, search using

 narnet greg

5. A quick example of an "o"penloop design:

 T   = simplenar_dataset;
 neto = narnet(1:2,10);
 [Xo,Xoi,Aoi,To] = preparets(neto,{},{},T);
 to = cell2mat(To); varto = var(to,1)% MSE reference
 Ntrials = 10
 rng('default')
 for i = 1:Ntrials
  neto = configure(neto,Xo,To);% Random weight init
  [ neto tro Yo Eo Xof Aof ] = train(net,Xo,To,Xoi,Aoi);
  %[ Yo Xof Aof ] = neto(Xo,Xoi,Aoi);
  % Eo = gsubtract(To,Yo);
  view(neto)
  NMSEo(i,1) = mse(Eo)/varto % Desire < 0.005
end

6. Modify to either save the net with the smallest normalized MSE or to break if NMSEo < 0.005 is encountered.

Post again if help is needed

Hope this helps.

Thank you for formally accepting my answer

Greg

[Gede Sukrisnha]

i have tried using narnet but it produce the same results, i just dont understand about open or closeloop, this is my first try of 1x120 target data 10 years from 1995 until 2014

% Solve an Autoregression Time-Series Problem with a NAR Neural Network
% Script generated by NTSTOOL
% Created Wed Feb 03 10:04:08 WST 2016
%
% This script assumes this variable is defined:
%
%   target1 - feedback time series.

targetSeries = tonndata(target1,true,false);

% Create a Nonlinear Autoregressive Network
feedbackDelays = 1:2;
hiddenLayerSize = 10;
net = narnet(feedbackDelays,hiddenLayerSize);

% Choose Feedback Pre/Post-Processing Functions
% Settings for feedback input are automatically applied to feedback output
% For a list of all processing functions type: help nnprocess
net.inputs{1}.processFcns = {'removeconstantrows','mapminmax'};

% Prepare the Data for Training and Simulation
% The function PREPARETS prepares timeseries data for a particular network,
% shifting time by the minimum amount to fill input states and layer states.
% Using PREPARETS allows you to keep your original time series data unchanged, while
% easily customizing it for networks with differing numbers of delays, with
% open loop or closed loop feedback modes.
[inputs,inputStates,layerStates,targets] = preparets(net,{},{},targetSeries);

% Setup Division of Data for Training, Validation, Testing
% For a list of all data division functions type: help nndivide
net.divideFcn = 'dividerand';  % Divide data randomly
net.divideMode = 'time';  % Divide up every value
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;

% Choose a Training Function
% For a list of all training functions type: help nntrain
net.trainFcn = 'trainlm';  % Levenberg-Marquardt

% Choose a Performance Function
% For a list of all performance functions type: help nnperformance
net.performFcn = 'mse';  % Mean squared error

% Choose Plot Functions
% For a list of all plot functions type: help nnplot
net.plotFcns = {'plotperform','plottrainstate','plotresponse', ...
  'ploterrcorr', 'plotinerrcorr'};

% Train the Network
[net,tr] = train(net,inputs,targets,inputStates,layerStates);

% Test the Network
outputs = net(inputs,inputStates,layerStates);
errors = gsubtract(targets,outputs);
performance = perform(net,targets,outputs)

% Recalculate Training, Validation and Test Performance
trainTargets = gmultiply(targets,tr.trainMask);
valTargets = gmultiply(targets,tr.valMask);
testTargets = gmultiply(targets,tr.testMask);
trainPerformance = perform(net,trainTargets,outputs)
valPerformance = perform(net,valTargets,outputs)
testPerformance = perform(net,testTargets,outputs)

% View the Network
view(net)

% Plots
% Uncomment these lines to enable various plots.
%figure, plotperform(tr)
%figure, plottrainstate(tr)
%figure, plotresponse(targets,outputs)
%figure, ploterrcorr(errors)
%figure, plotinerrcorr(inputs,errors)

% Closed Loop Network
% Use this network to do multi-step prediction.
% The function CLOSELOOP replaces the feedback input with a direct
% connection from the outout layer.
netc = closeloop(net);
[xc,xic,aic,tc] = preparets(netc,{},{},targetSeries);
yc = netc(xc,xic,aic);
perfc = perform(net,tc,yc)

% Early Prediction Network
% For some applications it helps to get the prediction a timestep early.
% The original network returns predicted y(t+1) at the same time it is given y(t+1).
% For some applications such as decision making, it would help to have predicted
% y(t+1) once y(t) is available, but before the actual y(t+1) occurs.
% The network can be made to return its output a timestep early by removing one delay
% so that its minimal tap delay is now 0 instead of 1.  The new network returns the
% same outputs as the original network, but outputs are shifted left one timestep.
nets = removedelay(net);
[xs,xis,ais,ts] = preparets(nets,{},{},targetSeries);
ys = nets(xs,xis,ais);
closedLoopPerformance = perform(net,tc,yc)

performance =

     6.3476e+03

trainPerformance =

     4.1384e+03

valPerformance =

     7.1433e+03

testPerformance =

     1.5616e+04

perfc =

     1.4618e+04

closedLoopPerformance =

     1.4618e+04

[Gede Sukrisnha]

see it has realy bad mse and error what should i do? i realy am a begginer in ann, and i realy in hurry cause of the deadline, dont have much time to learn nar in advance.

[Gede Sukrisnha]

nar is just have the target data, i just confused cause ann always have input data, thats why i avoided to use this narnet

[Greg Heath]

Divide your mse values by the average target variance to get NMSE, a normalized mse to help understand how good or bad your results really are. I have posted zillions of examples in the NEWSGROUP and ANSWERS.

Search on

 greg narnet NMSE

After taking a look at your data, I suggest you plot the autocorrelation functions of the three inner columns. You should be able to see that increasing the number of feedback lags should substantially improve your results.

In addition, take a look at the three cross-correlation functions between the 3 series.

Your data looks good. You should be able to get excellent results.

[Gede Sukrisnha]

how can i decide the number of target greg? is 1x120 matrices data should be enough to get the better results?

[Gede Sukrisnha]

the number of feedback lags u mean the number of delays?

[Greg Heath]

yes. No of delays and No. of hidden nodes have to be obtained by trial and error. Search the Newsgroup and Answers with

 Hmin:dH:Hmax Ntrials

Hope this helps

Greg

[Greg Heath]

I looked at the plots, autocorrelation functions and significant lags for all 3 series. I did not consider trn/val/tst datadivision so Ntrn = N = 360 with O =3 outputs yielding Ntrneq = 1080 training equations.

The plots looked noisy and could probably benefit from some smoothing (which I did not do!).

The autocorrelation functions of length N = 360 contained on the order of N/2 = 180 significant lags.

Since I did not use a validation set, I used narnet(1:d,H) with O =3 outputs and tried to minimize the number of unknown weights Nw = (d+1)*H+(H+1)*O subject to the following constraints

Nw < Ntrneq      % no overfitting  

MSE < 0.005*mean(var(t',1)) % NMSE < 0.005, Rsq > 0.995

Instead of a full-blown 3-loop search over delays d, hidden nodes H and number of random weight initializations, Ntrials, I first set an upperbound on H from the no overfitting upperbound condition

      H <= Hub = floor( (Ntrneq-O) / (d+1+O)) = 76

and considered, for d given

      Hmin ~ Hub/4, dH ~ 2, Hmax ~ Hub/2

I found that for d = 10, Ntrials = 10 that NMSE < 0.05 for 6 out of 10 trials when H = 26;

I hope this helps.

Note that you should use DIVIDEBLOCK for unbiased results.

Greg

[Rita]

Hi Greg, what happens if in addition to dividing data to 3 sets training /validation /test we also put this condition net.trainParam.goal = 0.005*var(targets,1); Thansks