Neural Network Times Series ahead prediction in Matlab, how to build input and output data

My goal is to predict N steps ahead with neuaral network in matlab. In order to do that first I train some part of the data and use trained values to predict the future behavior of it.

My question is based on Neural Network Times Series prediction on matlab. I was confused on how should I separate my Input and Output data and based on the difference between feedbackDelays and inputDelays.

As an Example: My Data is: [values are actually index values within the data] [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20] And I want to predict 5 point ahead of each 10 input which is:

[1,2,3,4,5,6,7,8,9,10] => [15]

[2,3,4,5,6,7,8,9,10,11] => [16]

[3,4,5,6,7,8,9,10,11,12] => [17]

and so on...

Matlab isn't like octave, I won't able to see the and analysis the Input data set and double check. In order to to this, how should my following code should be for function narxnet:

    **delay** = 5; % //Does delay means number of input??
    neuronsHiddenLayer = 50; 
    % Network Creation
    net = narxnet(1:delay,1:delay,neuronsHiddenLayer);  %=> Does is really do what I want to do as I decribed on above.

=====

X = S(1:length(S)-N); T = S(N:length(S));

for example length(S) is 5 X = [1,2,3,4,5] T = [6,7,8,9,10,11,12 ...]

I want to do that from the data of X if my input is *3*:

[1,2,3] => [6] //first data of T which is my output

[2,3,4] => [7] //second data of T which is from my output

=> On my code did I able to perform this solution, if not what should I do to achieve this solution?

=>If my output are in between [0-100], it predicts beyond that, larger values than 100, how it possible? I couldn't find where I make mistakes...

*My code:* My code come ups with a solution but I am not sure that it build the input and output values as I described above.

    S = load('newDataSet.txt');   %//sequence of numbers as time series.(attached)
    N = 5;                    %//5 points ahead will be predicted

    X = S(1:length(S)-N);      %//I think that it separates my input and output    
    T = S(N:length(S));

    X = X';
    T = T';
    X = num2cell(X);
    T = num2cell(T);

    %%2. Data preparation

     % Multi-step ahead prediction

    % Input and target series are divided in two groups of data:
    % 1st group: used to train the network

    inputSeries  = X(1:length(X)-N);
    targetSeries = T(1:length(X)-N);

    % 2nd group: this is the new data used for simulation. inputSeriesVal will 
    % be used for predicting new targets. targetSeriesVal will be used for
    % network validation after prediction

    inputSeriesVal  = X(length(X)-N+1:length(X));
    targetSeriesVal = T(length(X)-N+1:length(X)); % This is generally not available

    %%3. Network Architecture
    delay = 5;
    neuronsHiddenLayer = 50;
    % Network Creation
    net = narxnet(1:delay,1:1,neuronsHiddenLayer); *%1:1 correct for 1 output?*

    %%4. Training the network
    [Xs,Xi,Ai,Ts] = preparets(net,inputSeries,{},targetSeries); 
    net = train(net,Xs,Ts,Xi,Ai);
    view(net);
    Y = net(Xs,Xi,Ai); 

    % Performance for the series-parallel implementation, only 
    % one-step-ahead prediction
    perf = perform(net,Ts,Y);

    %%5. Multi-step ahead prediction

    inputSeriesPred  = [inputSeries(end-delay+1:end),inputSeriesVal];
    targetSeriesPred = [targetSeries(end-delay+1:end), con2seq(nan(1,N))];
    netc = closeloop(net);

    view(netc);

    [Xs,Xi,Ai,Ts] = preparets(netc,inputSeriesPred,{},targetSeriesPred);

    yPred = netc(Xs,Xi,Ai);

    perf = perform(net,yPred,targetSeriesVal);

    figure;

    plot([cell2mat(targetSeries),nan(1,N);
          nan(1,length(targetSeries)),cell2mat(yPred);
          nan(1,length(targetSeries)),cell2mat(targetSeriesVal)]')

    legend('Original Targets','Network Predictions','Expected Outputs')

Answer 1

[Greg Heath]

Perhaps you did not understand me:

 I do not understand what you want.

 An example using a MATLAB data would clarify the issue.

However, since you have submitted a single series example, there is only one function to use:

 help narnet
 doc narnet

A selection of effective feedback delays can be obtained from the significant values of the series autocorrelation function. For examples search using

 greg nncorr

Thank you for formally accepting my answer

Greg

[Avatar@ Wonderland]

Thank you for your answer.

My goal is to come up with a predictor that will able to predict (N) points ahead on my Single Series of data.In order to do that as I learned neural networks I need to train inputs by N point ahead of output. I have done it manually in octave to achieve this solution but I couldn't had a good result so I am trying to do it under matlab with it's functions that why they are new to me.

I couldn't solve and understand the way that how can I achieve the following situation. if I want to train my data as: (t is the time)

[t] [t+1] [t+2] [t+3] => [t+3+N]

[t+1] [t+2] [t+3] [t+4] => [t+4+N]

how can I achieve this with NARX or as you recommended narnet. please help me to figure out this situation. I have search your recommendations but they are like a black box to me to figure out how matlab's narnet function maintains the input and output's, and how can I force function to do what I want to do as I decribed above.

Thanks for further assistance.

Answer 2

[Greg Heath]

This is a poor data set for a realistic time-series example. Find a more appropriate example and code using

help nndatasets
doc nndatasets

help narxnet (or timedelaynet or narnet)
doc narxnet

Notice that there are various multi-dimensional examples for each of the three types of time-series.

 If you have problems, post relevant code with comments and error messages.

Thank you for formally accepting my answer

Greg

P.S. Search for relevant code using

 greg nncorr

[Avatar@ Wonderland]

Unfortunetly you didn't understand my question as the perspective I am asking. I just gave the example of values which are index values on the dataset; as a real example please see below:

newDataSet.txt: [33.25 42.42 42.75 43.33 ...]

Does program performs the my input and outputs like this: As you can see there are 4 input so dealy size should be 4. net = narxnet(1:delay,1:delay,neuronsHiddenLayer); % my output is only one output so should it be 1:1 instead of 1:delay.

and basically the idea for training should be as?

t1 t2 t3 t4   => t1803
t2 t3 t4 t5   => t1804
t3 t4 t5 t6   => t1805

so on... Training that I want to do, basically it slides:

(t-3) (t-2) (t-1)    (t)      => t+1800(1800 point ahead)
33.25  42.42  42.75  43.33     =>  40.00
42 42. 4.75   43.33  43.67     =>  41.32
42.75  43.33  43.67  44.33     =>  44.23
43.33  43.67  44.33  44.75     =>  52.56
43.67  44.33  44.75  44.92     =>  53.15
44.33  44.75  44.92  45.17     =>  41.67
44.75  44.92  45.17  45.75     =>  42.19
44.92  45.17  45.75  46.5      =>  42.98
45.17 45.75 46.5 46.67         =>  43.12

I am using the dataset that I am building that's why I was get confused about how could I build an efficient dataset and perform neural network. My dataset is independent from matlab's dataset but it is the same idea with time-series. I want to train my data as; the output value consist of N value ahead so when I test the weight in crosvalidation or test I can able to predixt N values ahead. Or do you have any suggestion that what is the best what to predict N step ahead prediction with my own data independent from matlab's.

[Avatar@ Wonderland]

Sorry on my previos respond I forget to add format for the code portion.

I have attached my data set.

As you suggested I searched "greg narnet closeloop" I was not able to find a example, there were one example, which works with narxnet. Also this code is focus to predict 1 point ahead. I don't know how to predict N point ahead.

I must need to predict N points ahead.

close all;
clear all;
clear all;

load('17HourTrace.mat'); %myKalman17
set_size = 1000;
targetSeries = myKalman17(1:set_size);  %it can be larger
targetSeries = targetSeries';

targetSeries_train = targetSeries(1:set_size * (4/5)     );
targetSeries_test  = targetSeries(set_size   * (4/5): end);

targetSeries_train = num2cell(targetSeries_train);
targetSeries_test = num2cell(targetSeries_test);

feedbackDelays = 1:4;
hiddenLayerSize = 10;

net = narnet(feedbackDelays, hiddenLayerSize);
net.inputs{1}.processFcns = {'removeconstantrows','mapminmax'};

[inputs,    inputStates, layerStates,targets] = preparets(net,{},{},     targetSeries_train);
[inputs_test,inputStates_test,layerStates_test,targets_test] = preparets(net,{},{},targetSeries_test);

net.trainFcn = 'trainrp';  % Levenberg-Marquardt
net.performFcn = 'mse';  % Mean squared error

net.plotFcns = {'plotperform','plottrainstate','plotresponse', ...
'ploterrcorr', 'plotinerrcorr'};

[net,tr] = train(net,inputs,targets,inputStates,layerStates);

outputs     = net(inputs_test,inputStates_test,layerStates_test);
errors      = gsubtract(targets_test,outputs);
performance = perform(net,targets_test,outputs)

view(net)

%  netc = closeloop(net);
%  [xc,xic,aic,tc] = preparets(netc,targetSeries,targetSeries); %%ERROR OCCURS
%  yc = netc(xc,xic,aic);
%  perfc = perform(net,tc,yc)

Thank you for your guidence and valuable time.

[Greg Heath]

The amount of time that you can predict ahead is limited by the length of the significant autocorrelation lags. If they are [ 1 3 5 7] and you want to predict 21 steps ahead it might be possible by using a multi-step closed loop approach.

So, your 1st task is to find the significant lags.

[Greg Heath]

IMPORTANT POINT:

All of the time series documentation uses default delays ID=1:2 and/or FD=1:2 with H = 10.

However, this is just a guess that it is a good place to start your search for a successful design. Personally, I would add 0 to ID (FD=0 is not allowed in a CL design);

For serious work you have to look at the significant correlation lengths. It is very unlikely that you can confidently predict beyond the longest SCL.

Hope this helps.

Greg