Data Science Asked by Carlos P Ceballos on February 4, 2021
I want to train a range of models, that are very similar; the convoluted layers are the same but the flattened part changes. Example; different density, initially I have model.add(Dense(512))
but I would also like to test 1000 and 4096; I would also like to test adding another layer.
I have no problem creating this models and running them, but it feels like a waste of time to retrain the convoluted layers every time when I’m not modifying anything in them.
Is it possible to save the training done in the convoluted layers and reuse that when training the flattened layers?
This is the base model I have:
model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), padding='same'))
model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), padding='same'))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), padding='same'))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), padding='same'))
model.add(Conv2D(128, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), padding='same'))
model.add(Flatten())
model.add(Dropout(0.2))
# -------------------- Start of Variable Section --------------
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.2))
# -------------------- End of Variable Section ----------------
model.add(Dense(1))
model.add(Activation('sigmoid'))
Other options I want to try (changes to be made inside the variable section) are:
model.add(Dense(1000, activation='relu'))
model.add(Dropout(0.2))
or
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.2))
etc.
To achieve that, where do I save the model? If I remove the flattened section:
model.add(Flatten())
model.add(Dropout(0.2))
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(1))
model.add(Activation('sigmoid'))
The model accuracy is unsurprisingly ~none, if I leave the section that is not variable, like this:
model.add(Flatten())
model.add(Dropout(0.2))
model.add(Dense(1))
model.add(Activation('sigmoid'))
I can save the model and load it, but how would I add those flattened layers?
You can simply train your convolutional layers, save the model and load weights from specific layers using the get_weights
and set_weights
methods (see also this previous answer). After loading the weights for you convolutional layers you can freeze those layers using the trainable
attribute to make sure the weights are not changed during training.
Answered by Oxbowerce on February 4, 2021
Get help from others!
Recent Questions
Recent Answers
© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP