My objective is to understand the keywords or phrases responsible for a sentence being assigned a particular class or sentiment. Say if I use a BERT model for emotion analysis,is it possible to identify the words or phrases for a sentence having a particular sentiment and changing which would alter their sentiment or semantic essence/meaning.
I wonder anyone could shed some light on this as to which layers could be exposed to identify those keywords or if there is any other method to do this.
It sounds to me like youâre trying to do style transfer from one sentiment to another. This is an active area of research currently.
One of the most popular papers is linked below but doesnât use transformers:
And here is a paper that builds on those methods using transformers:
Best of luck
Hi @FL33TW00D . Yes Thank you for sharing this paper the objective is to work towards that goal. I am currently trying to identify the weight of the words in the sentence to that particular sentiment/class. For example I am currently using the bart-large-mnli for zero-shot identification of a class(sentiment/intent) in a sentence.
eg: âThe service was outstanding but the breakfast was poorâ
for class labels âFoodâ or âqualityâ gives scores of 0.15 and 0.85 respectively. I want to identify the weight of the words in the sentence that contribute to that score of 0.85 for quality(Which primarily would be âserviceâ,âoutstandingâ,âbreakfastâ,âpoorâ). I want to know how I can extract this information assuming it is present in the final layers of the model.
Thank you for you help 
Hi @thomasdaryl,
Iâve seen a paper do what youâre looking for, as an extension of the DRG model they use the attention weights of the model in order to identify which tokens are contributing to the classification results most.
The paper is seen here:
They also have a helpful git repo here: GitHub - agaralabs/transformer-drg-style-transfer: This repository have scripts and Jupyter-notebooks to perform all the different steps involved in Transforming Delete, Retrieve, Generate Approach for Controlled Text Style Transfer
Which specifies that they used BERT to do exactly the task that youâre looking for as I understand it:
We have used BERT for classification. This classification trainings helps to find the attributes from the sentence. We choose one particular head of BERT model, for which the tokens which have high attention weights are those that are stylistic attributes of the sentence.
Hope this helps
Dear Thomas,
I have a question regarding your previous work âI am currently trying to identify the weight of the words in the sentence to that particular sentiment/classâ.
My task is simpler. I have NLP model that does sentiment analysis and I want to know word(s) that has the most influence on the assignment of sentiment (positive, negative, and neutral).
Did you publish any manuscript or code on your task?
Sincerely,
Oleksandr