How to use pretrain networks like vgg19,resnet18?

5 views (last 30 days)
I would like to use pretrained network like VGG19, RESNET18. The output of this network should be connected to LSTM.
Which layer should be considered to take output from pretrain network?
  2 Comments
Shilpa Sonawane
Shilpa Sonawane on 18 Oct 2022
i am working on the topic of lipreading. i have done preprocessing & have videos which consists of mouth region of person. I used VGG19 pretrain network. Now its output i would like to connect with LSTM network. I am confused from which layer of VGG19 the output is taken for furter processing. I referred many papers in which first pretrain netwrok is used and then its output is applied to LSTM network. LSTM network is used as input to the system is video which is frame sequence.

Sign in to comment.

Answers (1)

Nadia Shaik
Nadia Shaik on 23 Nov 2022 at 7:44
Hi Shilpa,
I understand that you wish to connect the output of CNN to LSTM network. MATLAB supports workflows containing both CNN and LSTM layers.
You can convert the videos to sequences of feature vectors using a pretrained convolutional neural network, such as GoogleNet, to extract features from each frame. And then, train an LSTM network on the sequences to predict the video labels.
Please check the link that contains an example showing the CNN+LSTM workflow https://www.mathworks.com/help/deeplearning/examples/classify-videos-using-deep-learning.html
Similarly, you can use the convolutional neural networks like VGG19, RESNET18 to extract features and then train an LSTM network.
I hope this helps

Categories

Find more on Deep Learning with Time Series and Sequence Data in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by