DeBERTa absolute Positions

Beginners
1

Hi! I was taking a look at Hugging Face’s DeBERTa implementation, but I couldn’t find where in the code are the absolute positions being added (they should come before the softmax layer), can someone point me towards the line where that is being done?
Thank you for your time :slight_smile:

1 Like
2

cc @lysandre maybe?

3

same here, wondering about absolute positions. Also @lysandre can you explain what the z_steps suppose to do in Debertav2.

Thanks a lot :slight_smile: