Neural network curve fitting: How to tell the net that some samples are more important than others?

Dear community,

I am using Matlab to evaluate a large set of physical measurements. What I am trying to do is a multidimensional curve fitting with the aid of the Neural Network toolbox in 2013a.

I am not a NN expert, so I mostly used the default values proposed by Matlab. I have a NN with 5 input variables, 1 output variable and 6864 samples. Basically, the NN does the fitting quite well with R=.997. This behavior is expected since it is known from physics that there is strong correlation between the 5 inputs and the output (but highly non-linear).

However, for some regions of the input variables, the NN results are not satisfying, as the differences between measurements and NN predictions are unreasonably high.

Now, I know from the measurements, that not all samples are equal in quality (measurement uncertainty, in this case). Thus, my straightforward approach was to assign some value of 'importance' to the input samples, meaning that those samples with low measurement uncertainty become 'more important' while samples with high meas. uncertainty are penalized.

I tried to find details about this sort of 'input weighting' in the User Guide and on google, but I have not found anything useful.

If you have any kind of alternative suggestions (using more than 1 hidden layer, etc.), please let me know. I am motivated to try everything :-)

Any help on my question is very highly appreciated! Thank you and regards, Philip

Answer 1

[Greg Heath]

Always begin by using as many defaults as possible and the help example

net = finet(5) % NO semicolon! Will reveal all of the defaults that do not have to be explicitly specified

If the help example data set is not relevant enough for your problem, choose an appropriate MATLAB data set so that we can compare numbers

help nndatasets help fitnet help train help mse

[x,t] = simplefit_dataset; net = fitnet(5); [ net tr y e ]= train(net,x,t,{},{},ew);

Vary ew and monitor the error e.

Hope this helps.

Thank you for formally accepting my answer

Greg

Answer 2

[Philip Ohnewein]

Dear Greg,

thank you very much for your hints and especially for the example code. This line of code does what I was looking for:

[ net tr y e ] = train(net,x,t,{},{},ew);

Strangely enough, however, I could not find this syntax being documented in any Mathworks document, only in your posts on MatlabCentral. I scrambled through 'doc train' and the NN toolbox reference pdf... But your syntax - which works perfectly fine! - is not described anywhere; I mean, what are the input and output arguments of train in this syntax...

Whatever the reason for the lacking documentation, you solved my problem, so thank you very much!

Philip

[Greg Heath]

This syntax was documented in 2004 when I started using MATLAB at home after I retired. Why the documentation was changed, I dunno. IIRC

[ net tr y e Xf Af ] = train( net, x, t, Xi, Ai, ew );

Sometimes looking at the source code helps. Sometime not.

type trainlm

type train

Answer 3

[Greg Heath]

See help mse and doc mse regarding the input: error weights.

Hope this helps.

Thank you for formally accepting my answer

Greg

Answer 4

[Philip Ohnewein]

Dear Greg,

thanks for your hint. I read the documentation of mse and it is basically what I was looking for. The problem is, I cannot get it to work as I expect...

Also, it is not clear to me whether Matlab uses mse for training (as a stopping criterion when the weighted mean square error is low) or only for post-processing (this is what it looks like on the help page...!?). In the users guide, there is an example for time series analysis, but not for curve fitting. I have not found anything useful via google, and my own experiments yield an error.

Here is my code

% calculate the weights
ew_base = .99999;
Re_max = max(Re_c(idx));
ew = ew_base .^ (Re_max - Re_c(idx))';

% set up the net + some options...
net = fitnet(5);
net.inputs{1}.processFcns = {'removeconstantrows','mapminmax'};
net.outputs{2}.processFcns = {'removeconstantrows','mapminmax'};
net.divideFcn = 'dividerand';  % Divide data randomly
net.divideMode = 'sample';  % Divide up every sample
net.divideParam.trainRatio = 60/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 25/100;
net.trainFcn = 'trainlm';  % Levenberg-Marquardt
net.performFcn = 'mse';  % Mean squared error

% tell the net that I would like to use the weights ew
% [net,tr] = train(net,inputs,targets);
[net,tr] = train(net,inputs,targets, ew);

And this is the error message which I get but do not really understand...

Error using network/train (line 272)
Number of input state timesteps does not match net.numInputDelays.
Error in dummy (line 18)
[net,tr] = train(net,inputs,targets, ew);

May I ask you to explain to me in some more detail how error weighting works in Matlab?

Thank your very much! Philip