Training feedforward neural network

I have to approximate the function Tnew=(9T1 + 8T2 + 4T3 + 4T4 + 2T5)/27, where T1,T2,T3,T4 and T5 are 13600-by-1 vectors (loaded from a given dataset). All the Ti's are functions of two variables X1 and X2 (X1 and X2 are also loaded from the same dataset). More specifically, I am asked to do the following:

1) Your dataset consists now of X1, X2 and T new. Draw 3 (independent) samples of 1000 points each. Use them as the training set, validation set, and test set, respectively. Motivate the choice of the datasets. Plot the surface of your training set.

2) Build and train your feedforward Neural Network: use the training and validation sets. Build the ANN with 2 inputs and 1 output. Select a suitable model for the problem (number of hidden layers, number of neurons in each hidden layer). Select the learning algorithm and the transfer function that may work best for this problem. Motivate your decisions. When you try different networks, clearly say at the end which one you would select as the best for this problem and why.

This is what I have so far:

Tnew=(9.*T1+8.*T2+4.*T3+4.*T4+2.*T5)./27;


trainset1=datasample(X1, 1000);
trainset2=datasample(X2,1000);
trainset3=datasample(Tnew, 1000);

valset1=datasample(X1, 1000);
valset2=datasample(X2,1000);
valset3=datasample(Tnew, 1000);

testset1=datasample(X1, 1000);
testset2=datasample(X2,1000);
testset3=datasample(Tnew, 1000);

plot3(trainset1,trainset2,trainset3), grid on;
title('Surface training set');
xlabel('X1'),ylabel('X2'),zlabel('Tnew');

alg1 = 'traingd';% First training algorithm to use
H = 50;% Number of neurons in the hidden layer
delta_epochs = [1,14,985];% Number of epochs to train in each step
epochs = cumsum(delta_epochs);

z=[trainset1';trainset2'];

net1=feedforwardnet(H,alg1);% Define the feedfoward net (hidden layers)
net1=configure(net1,z,trainset3);% Set the input and output sizes of the net
net1.divideFcn = 'dividetrain';% Use training set only (no validation and test split)
net1=init(net1);% Initialize the weights (randomly)

net1.trainParam.epochs=delta_epochs(1);  % set the number of epochs for the training 
net1=train(net1,z,trainset3);   % train the networks
a11=sim(net1,z) % simulate the network

net1.trainParam.epochs=delta_epochs(2);
net1=train(net1,z,trainset3);
a12=sim(net1,z)

net1.trainParam.epochs=delta_epochs(3);
net1=train(net1,z,trainset3);
a13=sim(net1,z)

However, this gives the following error:

Error using network/train (line 340)

Inputs and targets have different numbers of samples.

Error in personnalregression (line 33)

net1=train(net1,z,trainset3); % train the networks

Can anyone help me? Moreover, how do I incorporate the validation set in my program?

Any help would be appreciated!

Answer 1

[Shashank Gupta]

Hi Jente,

By my first glance of the code, I can see input is concatenation of transpose of trainSet1 and trainSet2 but output is simple trainSet3, I think the way you mentioned trainSet, the output should be transpose of trainSet3, even the error message says same that input and output sizes are not matching. I think this could be one possible reason of you getting that error. Also you can validate your trained model performance using perform function, Look at the below small piece of code for your reference.

% evaluation of model.
% Here x is input and net is trained model.
y = net(x)

% net is trained model, y is estimation and t is target
perf = perform(net,y,t)

I hope this helps.

Cheers

[Jente Bosmans]

Hello Shashank, thanks! I indeed needed to transpose trainSet3. Now, how can I validate my network on my validation set to tune my hyperparameters? Isn't the performing function used for my test set?