How to add attention map between words and tags

I am trying to take the advantage of semantic role labelling tags in NLI task. To do so, one of the method is to add semantic role labeling tags as special tokens and add attention between the words and its labels. How do I do it?