- Increase number of Layers
- Increase number of Parameters
- add an Activation layer to handle nonlinearity
- you might also give more iterate to network for learning , maxepoch=2000 since your data and network are small.
How to use deep learning for interpolation properly
5 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
Alexey Kozhakin
el 29 de Nov. de 2021
Comentada: Abolfazl Chaman Motlagh
el 5 de Dic. de 2021
Hi!
What I do wrong? I try to apply deep learning for interpolation function sin. And It learns not good. Insted of sin its just show line function. This is my code
clear all;
clc;
close all;
N=100;
T=2*pi;
x=0:T/(N-1):T;
y=sin(x);
%----- normalization ---------------
y=y+abs(min(y));
y=y./max(y);
x=x./max(x);
x=x;
y=y;
%---------- end normalization -----
plot(x,y,'o')
%------------------------
layers = [
featureInputLayer(1)
fullyConnectedLayer(10)
fullyConnectedLayer(1)
regressionLayer
];
options = trainingOptions('adam', ...
'MaxEpochs',20, ...
'MiniBatchSize',10, ...
'Shuffle','every-epoch', ...
'Plots','training-progress', ...
'Verbose',false);
[net inf] = trainNetwork(x',y',layers,options);
y1 = predict(net,x');
figure(2); hold on;
plot(x,y,'oblack','MarkerSize',20)
plot(x,y1,'x','MarkerSize',20)
0 comentarios
Respuesta aceptada
Abolfazl Chaman Motlagh
el 30 de Nov. de 2021
Hi,
The Sin(x) function is a complete nonlinear function, on the other hand your network is too simple to handle such a nonlinearity. for overcoming this problem you can:
here a simple solution for your model:
layers = [
featureInputLayer(1)
fullyConnectedLayer(20);
reluLayer;
fullyConnectedLayer(20);
reluLayer;
fullyConnectedLayer(20);
fullyConnectedLayer(1)
regressionLayer
];
the result is (5 fully connected layer with 20 parameters)
here are some other examples:
(2 fully connected layer each with 200 parameters) ( you can see still not converge correctly)
(3 fully connected layer each with 5 parameters)
so you can change this parameters (even change layers parameters indivisually) to see what is best option for your application.
6 comentarios
Abolfazl Chaman Motlagh
el 5 de Dic. de 2021
for first question, here is some example:
N=100;
T=2*pi;
x=0:T/(N-1):T;
y=sin(x);
y=y+abs(min(y));
y=y./max(y);
x=x./max(x);
layers = [
featureInputLayer(1)
fullyConnectedLayer(20);
reluLayer;
fullyConnectedLayer(20);
reluLayer;
fullyConnectedLayer(20);
reluLayer;
fullyConnectedLayer(20);
reluLayer;
fullyConnectedLayer(20);
fullyConnectedLayer(1)
regressionLayer
];
options = trainingOptions('sgdm', ...
'MaxEpochs',600, ...
'MiniBatchSize',10, ...
'Shuffle','every-epoch', ...
'Plots','training-progress', ...
'InitialLearnRate',0.01, ...
'LearnRateSchedule','piecewise', ...
'LearnRateDropFactor',0.1, ...
'LearnRateDropPeriod',200, ...
'Verbose',false);
[net inf] = trainNetwork(x',y',layers,options);
y1 = predict(net,x');
plot(x,y,'o','LineWidth',2)
hold on
plot(x,y1,'-','LineWidth',2)
legend('Sin(x)','Predicted')
Abolfazl Chaman Motlagh
el 5 de Dic. de 2021
for second question:
by putting relu function after one layer, all outputs pass over the relu function. so just by adding reluLayer in layers list, you are using this function on every neurons.
there are some standart implementation of other common activation function in matlab, like
- relu
- leakyRelu
- clippedRelu
- elu
- tanh
- swish
but if you want you can define your own activation function using matlab:
functionLayer:
or even complete Deep Network Layer structure and operation:
Más respuestas (0)
Ver también
Categorías
Más información sobre Define Shallow Neural Network Architectures en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!