Sentiment Analysis keywords

My objective is to understand the keywords or phrases responsible for a sentence being assigned a particular class or sentiment. Say if I use a BERT model for emotion analysis,is it possible to identify the words or phrases for a sentence having a particular sentiment and changing which would alter their sentiment or semantic essence/meaning.

I wonder anyone could shed some light on this as to which layers could be exposed to identify those keywords or if there is any other method to do this.

1 Like

It sounds to me like you’re trying to do style transfer from one sentiment to another. This is an active area of research currently.

One of the most popular papers is linked below but doesn’t use transformers:

And here is a paper that builds on those methods using transformers:

Best of luck

1 Like

Hi @FL33TW00D . Yes Thank you for sharing this paper the objective is to work towards that goal. I am currently trying to identify the weight of the words in the sentence to that particular sentiment/class. For example I am currently using the bart-large-mnli for zero-shot identification of a class(sentiment/intent) in a sentence.
eg: “The service was outstanding but the breakfast was poor”
for class labels ‘Food’ or ‘quality’ gives scores of 0.15 and 0.85 respectively. I want to identify the weight of the words in the sentence that contribute to that score of 0.85 for quality(Which primarily would be ‘service’,‘outstanding’,‘breakfast’,‘poor’). I want to know how I can extract this information assuming it is present in the final layers of the model.
Thank you for you help :smiley:

Hi @thomasdaryl,
I’ve seen a paper do what you’re looking for, as an extension of the DRG model they use the attention weights of the model in order to identify which tokens are contributing to the classification results most.
The paper is seen here:

They also have a helpful git repo here: https://github.com/agaralabs/transformer-drg-style-transfer
Which specifies that they used BERT to do exactly the task that you’re looking for as I understand it:

We have used BERT for classification. This classification trainings helps to find the attributes from the sentence. We choose one particular head of BERT model, for which the tokens which have high attention weights are those that are stylistic attributes of the sentence.

Hope this helps