Hey!
I’m trying to train a BART model with customized positional embeddings, similarly to what you have been doing and I have a few questions that you perhaps can help me with. First of all, say that I want to change the positional embeddings of BART to sinusoidal embeddings, just like you did @bengul, is that even possible? - My intuition is that we have to re-learn so many parts of the Transformer architecture that it might not be worth doing, or am I wrong here? My second thought is based on the assumption that it actually works, i.e. it is possible to change the positional embeddings, what kind of computing resource is necessary to be able to do this change in how the model treats the positions?
Hope you can help me with some insight in this 
@bengul @cowszero
Thanks!