Multiple softmax vectors in output layer of neural network using softmaxLayer
6 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
Isuru Rathnayaka
el 12 de Dic. de 2021
Comentada: Christian Holz
el 14 de Jun. de 2023
I'm using deep learning toolbox in MATLAB 2021a. And the neural network that I'm trying to build has multiple softmax vectors in output layer. (e.g. 10 softmax vectors of length 8). That is, the calculation is similar to how in-built softmax() function applies to each column of a matrix.
e.g.
>> a = randn(2,2)
a =
-1.1803 0.2963
1.6926 -0.1352
>> softmax(a)
ans =
0.0535 0.6062
0.9465 0.3938
However, I couldn't find a way to do this with softmaxLayer.
My code looks like this.
layersDNN = [
featureInputLayer(numInputs, 'Name', 'in')
fullyConnectedLayer(numInputs*2, 'Name', 'fc1')
batchNormalizationLayer('Name', 'bn1')
reluLayer('Name', 'relu1')
fullyConnectedLayer(numInputs*8, 'Name', 'fc2')
softmaxLayer('Name', 'sm1')
];
I'm trying to get the softmaxLayer to divide numInputs*8 nodes in last layer to numInputs vectors of length 8 and apply softmax function separately.
Alternatively I'm trying to remove softmaxLayer and apply softmax to reshaped output of network. Something like this.
lgraphDNN = layerGraph(layersDNN);
dlnetDNN = dlnetwork(lgraphDNN);
out1 = forward(dlnetDNN, X);
out2 = reshape(out1, [numInputs, 8]);
pred = softmax(out2);
% calculate loss, gradients etc.
I'm not sure if this is a good solution. I'd like to know if there's a way to do this using softmaxLayer, since the requirement doesn't feel like an extreme case.
2 comentarios
Abolfazl Chaman Motlagh
el 12 de Dic. de 2021
so what is the problem? did you face any error? did you test your last idea?
Respuesta aceptada
Prachi Kulkarni
el 10 de En. de 2022
Editada: Prachi Kulkarni
el 12 de En. de 2022
Hi,
From the R2021b release onwards, you can create numInputs number of fully connected layers, each with output size 8. Every fully connected layer can then be connected to its own softmax layer.
The outputs from the softmax layers can be concatenated using a concatenation layer and then passed on to the output layer.
2 comentarios
Christian Holz
el 14 de Jun. de 2023
Hello all,
a question regarding the proposed solution: were you able to implement it? Can you show the details?
Thank you
Más respuestas (0)
Ver también
Categorías
Más información sobre Image Data Workflows en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!