Extract most important words from model

Intermediate
1

Hi, I was wondering if it is possible to extract the most positive word or the most negative word from a sentence. Say, we have a movie review like, “This movie is amazing” and the most relevant word for sentiment classification would be the word “amazing”. Is it possible to use a trained BERT model to extract/return such most prominent words from a sentence?

Thanks.

2

Hi,

Yes that’s possible. There are some cool libraries out there that can be used to do that:

I believe it requires some tweaks to make it work on Transformer-based models, but I did this myself some time last year, so it’s definitely possible.

Edit: this Github thread might be helpful.

2 Likes
3

Thanks a lot for the quick response. The github thread has a lot of useful pointers.

Thanks!

4

Update, another useful package is Transformers-interpret.