TransWikia.com

How is it possible for RNN to do sentiment analysis?

Data Science Asked by WooseokChoi on December 12, 2020

I’m wondering how RNN can be used when doing sentiment analysis.

It seems that the characteristic of RNN is to remember what appeared in the past and determine the value of the present (future), but I think this RNN’s characteristic is useless when analyzing sentiment.

I am curious to see how it is possible to do emotional analysis by remembering the words that have appeared so far.

I think the way to train vectors that show how many times specific words appear is the only way to analyze sentiment.

If some RNN model train a sentence "good movie" and label "positive", can it predict a new sentence "movie good" as a "positive"?

Also, if a normal model(just fully connected deep neural network) train a sentence "good movie" and label "positive", is it hard for this model to predict a new sentence "movie good" as a "positive"?

3 Answers

It depends on how you configure your model and the layers you're using. You can add other layer's of DNN and experiment with those.

You should also consider using LSTM(Long short-term memory), since the past information do not get decayed quickly like in RNN, and you keep the present values, thus having a better model based on RNN.

You can also you a Bidirectional LSTM (BLSTM), which improve the learning process by using a cell state that runs through all your past data and generalize well for sentiment analysis.

Keep in mind that RNN cannot detect negation in sentences, like saying "This movie was not good", usually ANN need more configuration to understand those kind of aspects.

Another trick is to add a CNN layer to your model, so it can extract key words, thus when your data is forwarded to the RNN based layers, it will less noisy and more oriented to specific keywords.

Answered by MXK on December 12, 2020

RNNs do not learn to predict sentiment. They learn correlations between the input data and the target labels. If they see that every time the input contains the word "bad" they have to generate the label "negative", then they will learn it. If they see in the training data that the previous phenomenon happens always except when there is a "not" before "bad", then they will learn it. Depending on how the data distribution is, they may generalize the negation to any combination of verbs and adjectives, or maybe not, and they only handle negation appropriately when the input data is very very similar to the training data.

Your question seems to aim at understanding the "internal dynamics" of the RNN when it is predicting sentiment. While there is some research in that direction, I think they can still be considered as "black boxes", in that we do not actually understand the functions modeled by neural networks and therefore their outputs are not "explainable"

Answered by ncasas on December 12, 2020

Your problem is a sequence classification exercise. Yes, RNNs are very much needed to summarize the sentence intent as a whole. The last state of the RNN is fed to a Dense layer to predict the sentiment. This works nicely. You can further improve the exercise by adding 'Attention' by using all the hidden states of the RNN while predicting the output and paying attention to the ones that really matter.

See for e.g.: https://stackoverflow.com/questions/63060083/create-an-lstm-layer-with-attention-in-keras-for-multi-label-text-classification/64853996#64853996

Answered by Allohvk on December 12, 2020

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP