Using transformers (BERT, RoBERTa) without embedding layer

Wow this is an amazing response, thank you so much for this. I will need some time to digest it all, but this is exactly what I need!