Output embedding from each self-attention head from each encoder layer

Intermediate
1

Hi there!

I wanted the embeddings from each self-attention head of each encoder layer for one of my projects, is this possible with the hugging face library?

If not, can I just slice the original embeddings from each layer (suppose 768/12 = 128 size slice) to get the attention head output?

Thank You