In the deep learning custom layer, how to limit the learnable parameters?
3 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
Hello, everyone! I have designed a fuzzy system using deep learning toolbox, all the layers are designed using custom layer.
In the layers, some of my learnable parameters should not be zero, i.e. the standard deviation of the gaussian membership function. the fuzzify layer is shown as follows:
classdef fuzzify < nnet.layer.Layer
% fuzzify layer, the first layer of fuzzy system
% input : data, centers, sigmas
% output: the fuzzified value of each input data pair
%#codegen
properties (Learnable)
Centers
Sigmas
end
methods
function layer = fuzzify(Center,Sigma,name)
layer.Name = name;
layer.Centers = Center;
layer.Sigmas = Sigma;
end
function [Z,layer] = predict(layer,X)
% Define layer predict function here.
X = X';
[N,M] = size(X);
C = layer.Centers;
sigma = layer.Sigmas;
nRules = size(C,1);
Z = zeros(M*nRules,N,'like',X);
for i = 1:N
miu = exp(-(X(i,:)-C).^2./(2*(sigma.^2)));
Z(:,i) = miu(:);
end
end
function [Z,memory] = forward(layer,X)
% Define layer forward function here.
X = X';
[N,M] = size(X);
C = layer.Centers;
sigma = layer.Sigmas;
nRules = size(C,1);
Z = zeros(M*nRules,N,'like',X);
for i = 1:N
miu = exp(-(X(i,:)-C).^2./(2*(sigma.^2)));
Z(:,i) = miu(:);
end
memory=[];
end
function [dLdX,dLdW1,dLdW2] = backward(layer,X,Z,dLdZ,dLdSin)
% Define layer backward function here.
sigma = layer.Sigmas;C = layer.Centers;
X = X';
[N,M] = size(X);
nRules = size(C,1);
dLdX = zeros(M,N,'like',X);
dLdW1= zeros(nRules,M,'like',X);
dLdW2 = zeros(nRules,M,'like',X);
for i = 1:N
if isnan(sum(dLdZ(:,i)))
continue
end
miu = reshape(Z(:,i),nRules,[]);
dZdX = miu.*(-(X(i,:)-C)./sigma.^2);
dZdW1 = miu.*((X(i,:)-C)./sigma.^2);
dZdW2 = miu.*((X(i,:)-C).^2./sigma.^3);
a = reshape(dLdZ(:,i).*dZdX(:),nRules,[]);
dLdX(:,i) = sum(a);
if ~sum(~isfinite(dLdZ(:,i)))
dLdW1 = dLdW1+reshape(dLdZ(:,i).*dZdW1(:),nRules,M);
dLdW2 = dLdW2+reshape(dLdZ(:,i).*dZdW2(:),nRules,M);
end
end
end
end
end
In this code, the learnable parameter 'Sigmas' is the standard deviation of the gaussian membership function. When this parameter being updated, how can I set a threshold for it to prevented it becoming colse to 0.
what I want to do is that: after each iteration, check the value and replace the small number:
threshold = 1e-2;
layer.Sigmas(layer.Sigmas<threshold&layer.Sigmas>0)=threshold;
layer.Sigmas(layer.Sigmas>threshold&layer.Sigmas<0)=-1*threshold;
% this code is only an illustration to show what I want to do
How to realize this? if cannot, Is there any other way?
thanks so much
1 comentario
Johannes Pitz
el 16 de Nov. de 2022
Usually you would want to do something like: sigma = exp(parameter)
Or use the softplus function.
Respuestas (1)
Christopher Erickson
el 17 de Feb. de 2023
@Johannes Pitz' suggestion is excellent. I would add you could use sigmoid to also impose a maximum bound.
Good luck!
0 comentarios
Ver también
Categorías
Más información sobre Image Data Workflows en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!