Why peepholeLSTMLayer implemented in a tutorial is much slower than built-in lstmlayer?

2 visualizaciones (últimos 30 días)
Why this implementation of peepholeLSTMLayer https://au.mathworks.com/help/deeplearning/ug/define-custom-recurrent-deep-learning-layer.html is much slower than built-in lstmlayer?
What can be done to speed it up? For example, can it be compiled into a binary code?

Respuestas (1)

Hiro Yoshino
Hiro Yoshino el 31 de Ag. de 2023
I suppose that is because the implementation of interest is a custom model while the built-in LSTM is optimized for computation.
MATLAB has kept improving its performance over the years (see this). So I guess this is also the case with the buil-in capabilities in MATLAB.
As for speeding up, you may choose a CPU for computation of LSTM (See Tips).
You can also see this to speed up your custom trainings.
Hope these help you.
  1 comentario
Artem Lensky
Artem Lensky el 1 de Sept. de 2023
Editada: Artem Lensky el 1 de Sept. de 2023
Hi Hiro,
Thanks for the prompt reply. Yep, I train GRU/LSTM and PeepholeLSTM on a CPU. Peephole is not just slower, it's slower by a factor of 100 compared to standard LSTM. Luckily, this time I don't use custom training loops, it is trained by builtin train function. The model is extremly simple e.g. 1 layer with 8 PeepholeLSTM units. The dimensions of the input signals is 5 by (25k~30k).
1 'sequenceInputLayer' Sequence Input Sequence input with 5 dimensions
2 'rnn_1' peepholeLSTMLayer Peephole LSTM with 8 hidden units
3 'fc' Fully Connected 3 fully connected layer
4 'softmax' Softmax softmax
5 'classoutput' Classification Output crossentropyex
I ran the profiler (just 13 iteration of training) and see below what I've got. Any ideas how I can speed it up? Perhaps updating the tutorial code or compiling it to binary code e.g. mex. There must be something, it is just too slow. Thanks again!

Iniciar sesión para comentar.

Categorías

Más información sobre Image Data Workflows en Help Center y File Exchange.

Productos


Versión

R2023a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by