Adding noise to fitness neural network

Hi I have read, that noise can help improve the robustness of a neural network if it is applied correctly.

I have a network with 7 inputs, and three hidden layers with 12, 6 and 3 neurons respectively, I then have an output layer with 1 neuron.

This is a very simple extension of one of the tutorials:

trainFcn = 'trainlm';  % Levenberg-Marquardt backpropagation.
% Create a Fitting Network
l1 = 12; l2 = 6; l3 = 3; % number of neurons in each layer
hiddenLayerSize = [l1, l2, l3];
net = fitnet(hiddenLayerSize,trainFcn)
% Setup Division of Data for Training, Validation, Testing
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;
% Train the Network
[net,tr] = train(net,x,t);

I learned about processFcns when trying to normalise my inputs and noticing odd behaviour because minmaxmap was already being applied, scaling my features to the -1 to 1 range on a per column (feature) basis.

I am unsure how to add some random noise to the code above as I couldn't find any pre processing function that would enable it.

I would like:

• The noise to only be applied to training data, not the validation data.
• The noise to be added after the normalisation step so it affects each input irrelvant of the magnitude and range of the unnormalised features.

Because of these requirements I'm really unsure how to proceed, as a lot of these steps look to be automated.

Should I just set trainRatio to 100% and test the network's performance with an external call along the lines of afterwards?

% Test the Network
y = net(x);
e = gsubtract(t,y);
performance = perform(net,t,y)

If I did it this way I could make sure I do not apply the noise to the validation data. I would also have to remove the processFcns that normalizes the data, do it manually first, then apply my noise and then pass it to the train(net,x,t) with the automatic processFcns disabled.

Is there an easier way to do it? I would really like it if I could keep using Matlab's training panel which is great and does the training,testing and validation in a nice interface.

Thanks

Answer 1

[Gagan Agarwal]

Hi Julien,

I understand that you are trying to add noise to your neural network model to make your it more robust.

Here's a step-by-step guide to achieve this:

1. Normalize your data manually using the same normalization that the network would have applied.
2. Divide your data into training, validation, and testing sets.
3. Add noise to the normalized training data. You can generate noise using MATLAB's random number functions.
4. Train the network using the noisy training data.
5. Evaluate the network's performance on the validation and test sets, which were not affected by the noise.

I hope it helps!