# matlab neural network how to get similar r value for model and test

2 views (last 30 days)
Pangzhen Zhang on 31 Jul 2020
Commented: Pangzhen Zhang on 31 Jul 2020
Hi, Greg,
I am new to neural network and have read many of your post. My question is how to get similar R value for model and test for the neural network.
My data was 530 x 81 and target was 530x1, then I use Collinearity and stepwise VIF selection method (in R) to screen the input features and reduced to 530x48;
then the input was 48 x 530 and target was 1x530; I followed your suggestion on hidden neural selection to prevent overfit and the suitable range was 1-7 hidden neural for 1 hidden layer. (neural selection based on your previous post https://au.mathworks.com/matlabcentral/answers/347855-optimal-hidden-nodes-number);
With such small dataset, I used Bayesian Regularization algorithm with 70% for training and 30% for test, however, the problem is the R value for the training dataset and test are quite different. Train = 0.58, Test = 0.34 How could I get similar outcome for both train and test ?
I am aware that the training result looks not very good, but considering the nature of the dataset, the result was actuall very good if I can have both training and test R above 0.5.
netbr = fitnet(NeuronNo,'trainbr');
netbr.divideParam.trainRatio = 70/100;
netbr.divideParam.testRatio = 30/100;
netbr = train(netbr,T3,Price2);
br = netbr(T3);
perfbr = perform(netbr,br,Price2);

Behzad Navidi on 31 Jul 2020
It's been a long time since I work with neural network but I think If you change the size of training data to 80% and keep 20% for test you could achive more R value. Also try use "trainlm" instead of trainbr, it could make a difference.
trainFcn = 'trainlm'; % Levenberg-Marquardt backpropagation.
Also set different number of Hiden lyers
hiddenLayerSize = 10; % Size of the hidden layers in the network, specified as a row vector.
So:
net = fitnet(hiddenLayerSize,trainFcn);% net = feedforwardnet(param.hiddenSizes,param.trainFcn)
net.layers{1}.transferFcn='logsig'; % logsig or purelin
Or you can normalize your data before train and test it mybe help neural network to do better.
% Choose Input and Output Pre/Post-Processing Functions
% For a list of all processing functions type: help nnprocess
net.input.processFcns = {'removeconstantrows','mapminmax'};
net.output.processFcns = {'removeconstantrows','mapminmax'};
Pangzhen Zhang on 31 Jul 2020
thanks for the answer, I tried 80% and 20% it is a little bit better but still quite different between train and test dataset.
trainlm does not work well, likely due to the small dataset i use.
i also tried transferfuction and process function you mentioned, no improvement was observed. In fact, I tranform the input data was transformed first with some rows transformed to [-1,1], some other rows are binary data 0 or 1. I don't see any differences in the outcome with and without transformation.
I didn't try to increase the hidden layer as I think 1 layer should be sufficient as Greg mentioned in many of the posts.
So, the problem remains.
kind regards,
Pange