TransWikia.com

Keras, re use convoluted layers training for various flattened layers training

Data Science Asked by Carlos P Ceballos on February 4, 2021

I want to train a range of models, that are very similar; the convoluted layers are the same but the flattened part changes. Example; different density, initially I have model.add(Dense(512)) but I would also like to test 1000 and 4096; I would also like to test adding another layer.

I have no problem creating this models and running them, but it feels like a waste of time to retrain the convoluted layers every time when I’m not modifying anything in them.

Is it possible to save the training done in the convoluted layers and reuse that when training the flattened layers?

This is the base model I have:

model = Sequential()

model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), padding='same'))

model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), padding='same'))

model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), padding='same'))

model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), padding='same'))

model.add(Conv2D(128, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), padding='same'))

model.add(Flatten())
model.add(Dropout(0.2))
# -------------------- Start of Variable Section -------------- 
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.2))
# -------------------- End of Variable Section ----------------
model.add(Dense(1))
model.add(Activation('sigmoid'))

Other options I want to try (changes to be made inside the variable section) are:

model.add(Dense(1000, activation='relu'))
model.add(Dropout(0.2))

or

model.add(Dense(512, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.2))

etc.

To achieve that, where do I save the model? If I remove the flattened section:

model.add(Flatten())
model.add(Dropout(0.2))
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(1))
model.add(Activation('sigmoid'))

The model accuracy is unsurprisingly ~none, if I leave the section that is not variable, like this:

model.add(Flatten())
model.add(Dropout(0.2))
model.add(Dense(1))
model.add(Activation('sigmoid'))

I can save the model and load it, but how would I add those flattened layers?

One Answer

You can simply train your convolutional layers, save the model and load weights from specific layers using the get_weights and set_weights methods (see also this previous answer). After loading the weights for you convolutional layers you can freeze those layers using the trainable attribute to make sure the weights are not changed during training.

Answered by Oxbowerce on February 4, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP