How can I deal up with manual data division and multiple sequences of time series data?

Hello,

I'm working with Neural Toolbox using a dynamic network with time series data. Data consist on several contiguous data blocks (i.e. multiple sequences). Because there aren't so many large continous blocks, I thought I could make use of 'catsample' function to create a concurrent set of sequences, like described here: http://www.mathworks.es/help/toolbox/nnet/ug/bss36ff-1.html

The problem I'm encountering is related to the division of such data. I want to make a manual division into training, validation and testing data sets. So reviewing the data division functions avalaible on neural toolbox I chose the 'divideind' as the most suitable, but time series data are sequential data and therefore it's a cell array of double matrices which in turn each one has a concurrent set of sequences... So I can't imagine how to set up the indices for each data set. Because If I set for example:

[TrainInd, ValInd, TestInd] = divideind(nnInputData, 1:10, 11:20, 21:30);

I get TrainInd with all the sequences and ValInd and TestInd with nothing (just empty [])

Could you tell me some idea please?

Regards.

Answer 1

[Greg Heath]

Change your first input

help divideind

doc divideind

[Rafael]

Ok, I had misunderstood completely the first arg, I was thinking on it as the data structure :$. Thank you for noticing it.

Then I realize 'divideind' can't be used with multi-sequential data, isn't it? I need something which can differenciate the sequences of data, and not only the time steps.

[Greg Heath]

I think you have to use divideblock so that you

1. Train on 1:Ntrn for multiple candidate values of H (number of hidden nodes) and Ntrials of multiple random weight initializations for each value of H.

2 Choose the best design with the validation data Ntrn+1:Ntrn+Nval

3. Use the test set Ntrn+Nval+1:N to predict performance on unseen data.

[Torsten K]

Hello Rafael,

did you find a solution for dividing the complete sequences instead of dividing the time-steps? I would be very interested!

Best regards

Torsten