When forwarding sentence pair (sentence A, sentence B) through BERT (as shown below): [image] how to summarize the attention scores that each word of sentence B attends on each word of sentence A?

How to summarize the attention scores?

celsofranssa November 8, 2022, 10:21pm 1

When forwarding sentence pair (sentence A, sentence B) through BERT (as shown below):

how to summarize the attention scores that each word of sentence B attends on each word of sentence A?

Topic	Replies	Views
Extracting attention weights of summarization model Intermediate	425	August 12, 2021
How to extract attention gradients in bert Intermediate	612	April 16, 2022
Sentences' embeddings from BERT cross-encoder 🤗Transformers	271	December 22, 2022
Can I compare the attention of different encoder layers? Beginners	196	December 13, 2022
Is attention of different encoder layers comprabale? 🤗Transformers	266	December 6, 2022