Borrar filtros
Borrar filtros

Activations of freezed layers are different between before/after training, why?

3 visualizaciones (últimos 30 días)
I follow the example "transfer-learning-using-googlenet" where, the last 3 layers ('loss3-classifier','prob','output') are replaced with 3 new ones. Then I 'freeze' the first 141 layers (that is up to and including 'pool5-drop_7x7_s1'):
layers(1:141) = freezeWeights(layers(1:141));
lgraph = createLgraphUsingConnections(layers,connections);
Then I follow fine-tuning.
Since 'pool5-7x7_s1' is BEFORE 'pool5-drop_7x7_s1', I would expect that the following two vectors were the same:
b_orig= activations(net_orig, I, 'pool5-7x7_s1');
b_tune= activations(net_tune, I, 'pool5-7x7_s1');
but they aren't!... Any idea why?
p.s. I also tried the activation of several other layers BEFORE 'pool5-drop_7x7_s1', and I got different vectors.... 'I' is an image, 'net_orig=googlenet;', and 'net_tune' is the resulting net after tuning.
  2 comentarios
conngame el 15 de Jul. de 2018
I have the same problem using alexnet. Any explanations to this question?
ntinoson el 17 de Jul. de 2018
I did also try other CNNs, and same result. Activation of freezed layers after fine-tuning is different to that before fine-tuning (for the same input image of course). If anyone comes up with an explanation, drop a line!

Iniciar sesión para comentar.

Respuesta aceptada

Amanjit Dulai
Amanjit Dulai el 14 de Ag. de 2018
The vectors are different because when you fine tune on a new dataset, the average image in "imageInputLayer" is recalculated for your new dataset.
  2 comentarios
ntinoson el 27 de Ag. de 2018
ok, i see, thanks!
p.s. if 'normalization' was set to 'none' (i.e. NO data transformation applied, I guess the vectors would be the same, right?)

Iniciar sesión para comentar.

Más respuestas (0)


Más información sobre Image Data Workflows en Help Center y File Exchange.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by