TransWikia.com

how to reconstruct image from feature space of convolutional neural network?

Data Science Asked by Russell Butler on May 5, 2021

lets say i have fed an image into VGG19 pre-trained on imagenet as follows:

from tensorflow.keras.applications.vgg19 import VGG19
from tensorflow.keras.preprocessing import image
from tensorflow.keras.applications.vgg19 import preprocess_input
from tensorflow.keras.models import Model
import numpy as np

base_model = VGG19(weights='imagenet')
model = Model(inputs=base_model.input, outputs=base_model.get_layer('block5_pool').output)

img_path = 'C:/shared/a5/images/training/n01518878_8432.JPEG'
img = image.load_img(img_path, target_size=(224, 224))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)

features = model.predict(x)

i then extract the values at layer5 max pool layer, which are determined by the input image, and unique for every different image that is fed into the network.

does anyone know how to reconstruct the original image based on the features extracted from a specific layer? thank you

One Answer

One choice is to train a neural network model to take these values and output original images.

Notice that usually some data is loss in this process so it might be impossible to reconstruct the image with perfection.

You could try inverting the functional form but:

  • CNNs usually use ReLu activation which is not bijective.
  • Pooling layers throws information away (it still could be fixed if we had redundant information)
  • The weight matrix might not be inversable

Answered by Pedro Henrique Monforte on May 5, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP