Data Science Asked by Howard Wang on July 31, 2020
I’m trying to train a search and item encoder and this is the model I have
input_search = Input(shape=(40,), dtype='int64', name='input_search')
input_title = Input(shape=(40,), dtype='int64', name='input_title')
input_desc = Input(shape=(40,), dtype='int64', name='input_desc')
input_brand = Input(shape=(40,), dtype='int64', name='input_brand')
embedding = Embedding(input_dim=20000, output_dim=50, input_length=40)
s_emb = embedding(input_search)
t_emb = embedding(input_title)
d_emb = embedding(input_desc)
b_emb = embedding(input_brand)
s = GlobalMaxPool1D()(s_emb)
t = GlobalMaxPool1D()(t_emb)
d = GlobalMaxPool1D()(d_emb)
b = GlobalMaxPool1D()(b_emb)
concat = concatenate([t, d, b])
concat = Dense(128)(concat)
s = Dense(128, name='vec')(s)
similarity = Lambda(cos_sim)([s, concat])
model = Model(inputs=[input_search, input_desc, input_brand, input_title], outputs=similarity)
_______________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_search (InputLayer) [(None, 40)] 0
__________________________________________________________________________________________________
input_title (InputLayer) [(None, 40)] 0
__________________________________________________________________________________________________
input_desc (InputLayer) [(None, 40)] 0
__________________________________________________________________________________________________
input_brand (InputLayer) [(None, 40)] 0
__________________________________________________________________________________________________
embedding_1 (Embedding) (None, 40, 50) 1526100 input_search[0][0]
input_title[0][0]
input_desc[0][0]
input_brand[0][0]
__________________________________________________________________________________________________
global_max_pooling1d_5 (GlobalM (None, 50) 0 embedding_1[1][0]
__________________________________________________________________________________________________
global_max_pooling1d_6 (GlobalM (None, 50) 0 embedding_1[2][0]
__________________________________________________________________________________________________
global_max_pooling1d_7 (GlobalM (None, 50) 0 embedding_1[3][0]
__________________________________________________________________________________________________
global_max_pooling1d_4 (GlobalM (None, 50) 0 embedding_1[0][0]
__________________________________________________________________________________________________
concatenate_5 (Concatenate) (None, 150) 0 global_max_pooling1d_5[0][0]
global_max_pooling1d_6[0][0]
global_max_pooling1d_7[0][0]
__________________________________________________________________________________________________
search (Dense) (None, 128) 6528 global_max_pooling1d_4[0][0]
__________________________________________________________________________________________________
product (Dense) (None, 128) 19328 concatenate_5[0][0]
__________________________________________________________________________________________________
lambda (Lambda) (None,) 0 search[0][0]
product[0][0]
==================================================================================================
Total params: 1,551,956
Trainable params: 1,551,956
Non-trainable params: 0
__________________________________________________________________________________________________
Items have 3 features – title, description, and brand. I want to use the same embedding layer between all 4 inputs to generate embeddings in the same vector space. However, for prediction time, I would like to remove the 3 inputs for the item, and make the output of the model the embedding vector from the dense layer named ‘vec’ to store the feature vectors. This is what I have tried:
search_model = Model(model.inputs[0], model.layers[-2].output)
item_model = Model(inputs=model.inputs[1:], outputs=model.layers[-1].output)
but I get the error
ValueError: Graph disconnected: cannot obtain value for tensor Tensor("input_brand_1:0", shape=(None, 40), dtype=int64) at layer "input_brand". The following previous layers were accessed without issue: []
Is there a way for me to share the embedding layer weights between the search and the item features that would allow me to prune certain layers come inference time? Or perhaps create 2 separate embedding layers, where 1 is run through the search and the other is run through the 3 features of the item but somehow keep the weights in both layers the same?
Get help from others!
Recent Questions
Recent Answers
© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP