Will XLM-R-(X)XL be available in Models?

Models
1

Hi guys,

PyTorch FairSeq has recently published larger XLM-R models here: fairseq/examples/xlmr at master · pytorch/fairseq · GitHub

They have XLM-R-XL and XLM-R-XXL models that are much larger than the XLM-R-Base and XLM-R-Large that are currently available in :hugs: Models. Is it possible to load these models in :hugs: Transformers? What would it take to convert them?

2

I know this is an old post, but in case you are still interested there is an ongoing pull request for it here: Add support for XLM-R XL and XXL models by modeling_xlm_roberta_xl.py by Soonhwan-Kwon · Pull Request #13727 · huggingface/transformers · GitHub

1 Like