Testing neural network's forecasting ability

Hello,

I'm quite new to neural networks, but I'm currently using them to forecast demand per month of a production company. I've build a neural network using the nnstart command and I chose the time series option. The network building, testing and validation is working great, and I can plot a graph of the outcomes of the ANN as well as the "feedback" matrix, which contains the true values. However, I want to test the forecasting abilities by letting the network predict new values WITHOUT RE-TRAINING the network!

I've tried certain things already, but I just cannot get it to work yet. Does anyone have a solution for my problem? In short:

- I built a network, and tested it with data from 3 years

- I want to see how the network forecasts the first 6 months of the 4th year, data that is not included in the training data

Thanks in advance, as I said I'm quite new to Neural Networks so maybe the solution is very easy.

[Greg Heath]

Insufficient information.

Which net?

What delays and hidden node size?

input matrix size?

target matrix size?

trn/val/tst ratios?

Answer 1

[Greg Heath]

Your code works on the dataset in the documentation

 help narnet
 doc  narnet

However, I did not try it on any of the other NARNET example datasets:

 help nndatasets
 doc nndatasets

I can only quote my general approach :

a. Test code on one or more a MATLAB example datasets

b. Use your data to obtain the best openloop performance of multiple (>=10) designs

c. If closing the loop sufficiently degrades performance, then train netc.

Hope this helps

Thank you for formally accepting my answer

Greg

[Timen]

Ok thank you, I will try that!

Answer 2

[Greg Heath]

Your so called "feedback" matrix is the "target" matrix. It is used in the nar and narx nets for openloop design(training and validation)and testing.

To forecast beyond your original data, you have to go into the closeloop mode. The output matrix then becomes the feedback matrix.

I have many posts in the NEWSGROUP and ANSWERS you can reference. Search on

 greg closeloop

Hope this helps.

Thank you for formally accepting my answer

Greg

[Timen]

Dear Greg,

First of all; thank you for your quick reply! I've looked at the posts but I still cannot get it to work. If I change my function towards a netc function, I get a slightly increasing line for the first n=20 samples, after that the line is just a horizontal line. However, the datapoints in which the line is increasing are already known, so there's no point in trying to predict these values; I want to predict values that are not in the data set yet. I'll paste my code below (without all the comments Matlab added, to improve readability):

 T = tonndata(SATrend,false,false);
trainFcn = 'trainlm';  % Levenberg-Marquardt

 feedbackDelays = 1:2;
hiddenLayerSize = 10;
net = narnet(feedbackDelays,hiddenLayerSize,'open',trainFcn);

 [x,xi,ai,t] = preparets(net,{},{},T);

 net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;

 [net,tr] = train(net,x,t,xi,ai);

 y = net(x,xi,ai);
 e = gsubtract(t,y);
performance = perform(net,t,y)

 view(net)

 netc = closeloop(net);
[xc,xic,aic,tc] = preparets(netc,{},{},T);
yc = netc(xc,xic,aic);
perfc = perform(net,tc,yc)

I've also uploaded a graph of the data; the red line is the "actual" data, the green line is the prediction of the open loop NAR network (which is quite accurate) and the blue line is the prediction of the closed-loop network, with the horizontal line from n = 20.

Thanks in advance! I hope you can help me and I'm not asking too detailed questions!