Will XLM-R-(X)XL be available in Models?

Hi guys,

PyTorch FairSeq has recently published larger XLM-R models here: fairseq/examples/xlmr at master 路 pytorch/fairseq 路 GitHub

They have XLM-R-XL and XLM-R-XXL models that are much larger than the XLM-R-Base and XLM-R-Large that are currently available in :hugs: Models. Is it possible to load these models in :hugs: Transformers? What would it take to convert them?

I know this is an old post, but in case you are still interested there is an ongoing pull request for it here: Add support for XLM-R XL and XXL models by modeling_xlm_roberta_xl.py by Soonhwan-Kwon 路 Pull Request #13727 路 huggingface/transformers 路 GitHub

1 Like