Suggestions to improve my Neural Network?

Hi!

I am working with 3 files that have over 30,000 rows corresponding to signals from a sensor [Data is (30,000 x 2,000)]

I want to classify the signals that are abnormal -- it tends to occur across the 3 files around the same group of columns but this abnormality is 7% of the data.

My approach so far has been to detrend the three files, normalize and combine the 3 files in a series so now I have 30,000 x 6,000. (Not sure how I would do it if I had to use the three files separately to train).

I have a separate file with the class of 0 and 1 (1 being that the row is abnormal).

The results so far have been as follows when training with the imbalanced data:

I also tried down-sampling the class 0 data to get it to 2:1:

When trying the model on another different dataset, the result is not so great for the output class 1 row:

My code is a modification from the App and from what I've found from the community so it might not be the best:

% Solve a Pattern Recognition Problem with a Neural Network
%
% This script assumes these variables are defined:
%
%   Training - input data.
%   Joint_index - target data.

x = Training';
t = Joint_index';

% Choose a Training Function
trainFcn = 'traincgf';  % Scaled conjugate gradient backpropagation.


% Create a Pattern Recognition Network
hiddenLayerSize = 50;
net = patternnet(hiddenLayerSize, trainFcn);
net.trainParam.lr = 0.0045;
net.trainParam.epochs = 500;
net.trainParam.goal = 1e-6; 
net.trainParam.max_fail = 500;

% Setup Division of Data for Training, Validation, Testing
net.divideFcn = 'dividerand';  % Divide data randomly

% Setup Division of Data for Training, Validation, Testing
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;

% Train the Network
[net,tr] = train(net,x,t);

% Test the Network
y = net(x);
e = gsubtract(t,y);
performance = perform(net,t,y)
tind = vec2ind(t);
yind = vec2ind(y);
percentErrors = sum(tind ~= yind)/numel(tind);

To test on a separate dataset:

x = (Training2(1:8064,:))'; % train data
t = (Joint_index2(1:8064,:))'; % target

% Test the Network
y = net(x);
e = gsubtract(t,y);
performance = perform(net,t,y)
tind = vec2ind(t);
yind = vec2ind(y);
percentErrors = sum(tind ~= yind)/numel(tind);

% Recalculate Training, Validation and Test Performance
trainTargets = t .* tr.trainMask{1};
valTargets = t .* tr.valMask{1};
testTargets = t .* tr.testMask{1};
trainPerformance = perform(net,trainTargets,y)
valPerformance = perform(net,valTargets,y)
testPerformance = perform(net,testTargets,y)

% Plots
figure, plotconfusion(t,y)

Answer 1

[Ayush Modi]

Hi Wookie,

As per my understanding, you would like to know ways you can increase the accuracy of the model.

I observed that you tried down sampling class 0 to 2:1 ratio. But you couldn't do it. One of the reasons could be that the model is not getting enough data to train the weights after down sampling.

You can try augmenting class 1 data instead. This will maintain the ratio of both classes and provide the model with larger number of sample points to train.

Please try to check the below mentioned methods also to increase the accuracy of the model:

• Dropout layers (To avoid overfitting)
• Random over-sampling
• Random under-sampling

I hope this resolves the issue you were facing.