Does groupedCon​volution2d​Layer support input data with T dimension.

1 visualización (últimos 30 días)
Artem Lensky
Artem Lensky el 5 de Sept. de 2023
Editada: Artem Lensky el 18 de Sept. de 2023
Whenever I apply groupedConvolution2dLayer to data with T-dimension e.g.
groupedConvolution2dLayer([1 filterSize], 1, "channel-wise", DilationFactor=dilationFactor, Padding="same", Name="conv_1_" + k + "_" + l)
the following error is produced
I could implement channel-wise convolution using the conncatenation depthConcatenationLayer, but such networks end up being much much slower to train.

Respuestas (1)

Milan Bansal
Milan Bansal el 13 de Sept. de 2023
I understand that you are facing an error while using 'groupedConvolution2dLayer' while passing input data with "T" dimension.
Output from the layer "channels_1_1" which act as input to the layer "conv_1_1_1" has dimension 5(C) 1(B) 128(T), that means it is a vector-sequence data, with no "spatial(S)" dimensions.
Where as, the "groupedConvolution2dLayer" expects an image data with "spatial(S)" dimensions and channel(C) dimension for convolution. It does not support sequence input hence It does not support "Time(T)" dimension.
Refer to the documentation link to know more about dimension labels in "dlarray".
Use "finddim" function to find the dimension and labels of "dlarray".
dim = finddim(layout,"S")
Refer to the document link to know more about "groupedConvolution2dLayer".
  3 comentarios
Ben el 18 de Sept. de 2023
It looks like we don't support sequence inputs to groupedConvolution2dLayer but it seems like dlconv does support grouped convolution on sequence data, so it might be reasonable to write a custom layer for this. Here's a demonstration of dlconv on sequence data - note the usage of WeightsFormat :
C = 6; B = 1; T = 128;
X = dlarray(randn(C,B,T),"CBT");
filterSize = 15;
channelsPerGroup = 2;
filtersPerGroup = 4;
numGroups = 3;
W = randn(filterSize,channelsPerGroup,filtersPerGroup,numGroups);
b = zeros(filtersPerGroup*numGroups,1);
Y = dlconv(X,W,b, WeightsFormat="TCUU");
% alternative to WeightsFormat is to make W a formatted dlarray:
W = dlarray(W,"TCUU");
Y = dlconv(X,W,b);
You should be able to use this in the implementation of a custom layer's predict function to get a grouped convolution over sequence data.
The relabel T to S strategy could be OK if you pad the data accordingly. You'd have to do the padding manually yourself, if using trainNetwork it could be quite awkward to do this such that you have the minimal amount of padding necessary per-minibatch (I think you'd need to write a custom datastore). Alternatively you could do that in the "minibatch function" of minibatchqueue and use a custom training loop.
I would have suspected the reason manually splitting the channel dimension is slow is because instead of doing one big grouped convolution the software has to do many separate convolutions. If depthConcatenationLayer is itself very slow then that's something we should look at internally.
Artem Lensky
Artem Lensky el 18 de Sept. de 2023
Editada: Artem Lensky el 18 de Sept. de 2023
Thank you Ben. I will play with dlconv and the suggested implementation.
You might be right about the cause for the slow execution, it is not depthConcatenationLayer per se but rather the fact that now convolution is performed on each channel separately and no optimision can be perfromed on all channels altogether.

Iniciar sesión para comentar.


Más información sobre Image Data Workflows en Help Center y File Exchange.




Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by