TransWikia.com

What are the equations involved in calculation of the parameters of embedding layer?

Data Science Asked by Harish Reddy on December 20, 2020

I’m trying to perform sentiment analysis on some data using keras.I’m using embedding layer and then LSTM. I know that embedding layer decreases the sparsity of the one hot encodings of the words and its parameters are trained while back-propagation, but I don’t know the mathematics of its implementation.

Thanks in advance.

One Answer

Like all hidden layers in a neural network, an embedding layer can be thought of as a feature extractor with parameters that are automatically learned during training. So, like any other layer, the parameters are adjusted during training by the backpropagation algorithm.

The specific equations used to compute error gradients and update weights are likely to depend on the optimizer used to train the network. The classic optimizer is stochastic gradient descent, but more sophisticated optimizers like Adagrad, RMSProp, and ADAM are commonly used nowadays.

Answered by zachdj on December 20, 2020

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP