RoBERTa/BERT for Norwegian Currently, there is only a very limited amount of BERT-like models for Thai on the hub: Hugging Face – The AI community building the future. . For this project, the goal is to create a RoBERTa/BERT model for just the Norwegian language. Model A randomly initialized RoBE…

PReTrain RoBERTa from scratch in Norwegian

saattrupdan June 28, 2021, 5:39pm 3

It could make sense to pretrain a common Scandinavian model instead, as our languages are so similar. I’ve made a project suggestion here: Scandinavian RoBERTa.

2 Likes

Topic		Replies	Views
PreTrain RoBERTa from scratch in Thai Flax/JAX Projects	3	639	July 2, 2021
PreTrain RoBERTa from scratch in Hindi Flax/JAX Projects	24	2027	December 10, 2021
PreTrain RoBERTa from scratch in Portuguese Flax/JAX Projects	16	2355	October 4, 2021
PreTrain RoBERTa from scratch in Indonesian Flax/JAX Projects	6	533	June 28, 2021
RobIt : PreTrain RoBERTa-base from scratch in Italian Flax/JAX Projects	4	470	June 29, 2021

PReTrain RoBERTa from scratch in Norwegian

Related topics