NARX Multi step ahead predictions

I am relatively new to neural network and I have a few, perhaps very basic questions that I need some help to better understand it.

1) I am trying to use NARX to predict multi-step ahead time series. I have 4 inputs (say 500 rows by 4 column) and 1 target (say 500 rows by 1 column) output historical data and I would like to predict say 10 days ahead. I am using Narx because I think it suit me the best. However, I could hardly get my head around with the multiple step ahead prediction because (please correct me if I am wrong), from what I understand I will need inputs for the future 10 days in advance to be fed into the network in which I do not have. So how can I get around with this? I don't think NAR will suit in this case because I need these inputs and these are the key characteristics of the time series. Or should I use multiple model? something is missing...

2) after the training using open loop, I switched to close loop however I get very bad performance (mse open loop of ~0.3 and close loop ~250) checked both error correlations, looks ok within 95% confident limit, any hints on what have gone wrong would be very much appreciated.

3) I generated the simple code for the network, may I know how should I use it to forecasting time series? I don't think I will need to keep on running the training data set and test data set every 10 days (in this example of 10 days ahead predictions) or that is how it works?

I have been searching the forum quite thoroughly for these answers but I couldn't find the answer I am looking for... any help would be greatly appreciated.

Answer 1

[Greg Heath]

1. Traditionally, with

 [ I N ] = size(input)  % [ 4 500 ]
 [ O N ] = size(target) % [ 1 500 ]

you would use a subset of the statistically significant delays from the

 a. target/input crosscorrelation functions. 
 b. target autocorrelation function

I have posted many relevant examples in both the NEWSGROUP and ANSWERS. Try the search words

 nncorr significant greg

2. However, before doing 1, you might just cross your fingers and try ("o" and OL will indicate an open-loop net)

 ID = 0:10; FD = 1:10; 
 neto           = narxnet(ID, FD, H);
 neto.divideFcn = 'divideblock';

and vary the number of hidden nodes, H; If needed, the default 0.7/0.15/0.15 trn/val/tst ratios could be modified to increase the size of the training set.

3. Yes, you need inputs for post target NARX prediction. If you have none, try to estimate them using the original input and target data.

 NOTE: THIS IS NOT MENTIONED IN MATLAB DOCUMENTATION !

4. I have found that just closing the loop on an OL net only works if the OL error is very small. I try to obtain at least

MSEtrno <= 0.001*mean(var(target',1))    % i.e., Rsqtrno >= 99.9%

However, the reliability of this assumption remains to be investigated.

5. The fastest way to obtain help from this forum is to use the MATLAB example data obtained from

 help nndatasets
 doc  nndatasets.

Hope this helps.

Thank you for formally accepting my answer

Greg

[Teck Kong Chong]

Hi Greg, Thank you so much for your answers, really appreciated that!

I am using net.divideFcn = 'divideblock' with a ratio of 60/20/20 and 'trainbr' for my Narx. I am yet to find out your recommendation about nncorr and significant. I have a few other questions in my head, hopefully you would be able to help me understand them better, apology if the questions are trivial:

1) I have got a much better mse by using your recommendation setting ID = 0:10, FD = 1:10 and varying the # of neurons. After finding the so-called optimal neuron number, I reset the ID = 1:10. It also works fine both OL and CL. However, I do not get the idea behind the use of '0' instead of '1' to get the neuron number, can you briefly explain to me or point me to a right direction please?

2) Why do you choose 10 as maximum delay and how '0' helps in this case?

3) How do I predict say 3 steps in advance (e.g. predict Target for 3rd day in advance), I assumed we can only use 'A step ahead prediction'. I have seen someone posted in another forum said that it can be done by adjusting the ID and FD, such that ID = #step_ahead:Delay+#step_ahead, and ID=FD, where #step_ahead = 3 in this case and delay is network delay. Is this a correct implementation? I tested this but doesn't seem to work in my case, am I missing something?

4) My one step ahead prediction plot shifted to the left by the amount of delay and I don't think this is correct. Is there a remedy to fix this? I know I can adjust the plot to move the prediction point shift to the right but I am not sure if what I did is correct?

5) In the nntraintool gui, there is "effective # param", "Gradient" and "Mu", can you briefly explain what exactly they are (I know about gradient) and how the numbers are calculated? can it be adjusted manually?

6) Last;y, do you have any easy to read recommendation about the relationship between ID vs Input/target error-correlation and FD vs Target Auto-correlation. I know you mentioned in a separate threads to look for the peaks beyond '0' lag. Do you meant that if I have another peak (based on the 'ploterrcorr') at say 20 then I will have to set my FD lag to 20? it is the same for 'plotinerrcorr' too?

Thank you very much for your time!

[Greg Heath]

I don't remember seeing this reply (I was on vacation).

0. a. Why trainbr with 60/20/20?

0.b. trainbr doesn't use the val set. Check tr to find the division it gave you.

1. ID = 0 is similar to fitnet. Typically, lag = 0 has the highest input/target correlation. May have been overlooked by MATLAB. Don't have to throw it away.

2. Just investigating defaults.

3. The only way I see it is to remove delays 0:2 and change your mental time reference, so that the current output is really the future. I guess you can use MATLAB's removedelay function, However, it's smoke-and-mirrors as far as I am concerned.

4. I dunno...my brain is beginning to hurt!

5. That is all trainbr stuff. You'll probably have to explore external references after checking trainbr with the help, doc and type commands.

6. What I have posted I made up using common sense. You don't have to cover every peak in a correlation plot ... your design is trying to minimize an average based error criterion. However if the input/error correlation has large peaks coinciding with a gap in ID and/or FD, I'm sure you can fiddle with ID and FD to see what you can do. Uncharted waters for me.

7. My current goal is to have a foolproof way to obtain a good CL net by closing the loop on a good OL net. Training after closing is only working sometime. My current thought is it may be an overfitting problem.

Hope this helps.

Greg