I am working on attention analysis. I want to learn more about where self attention made mistakes while attending to context query. Given two sentences, I am interested in learning more about where self-attention should have paid more attention (and not irrelevant tokens) to provide correct answers. In general, what went wrong in processing a given sample even if fine-tuned transformer is employed.
While there are projects based on visualization like
ExBERT, I am not sure if it’s straightforward to extract the information I’m looking for.
Do you know of any good projects, or workarounds in
Transformers to answer my query ?