How to create word embeddings for non-English languages using BERT-like models?

ekdnam · March 22, 2021, 7:05pm

Hi all! This is my first topic here, so apologies in case I make some errors.

Currently I am working on creating custom word embeddings for an Indian language, Marathi. They will be later used for creating a NMT model for translation between Marathi and English.

How to do so using transformers? Also, what is the required data cleaning process?

Topic		Replies	Views
Creating word embeddings using BERT of machine generated sequential data Models	0	256	April 7, 2023
Using custom embeddings for pre-training model for new vocabulary Beginners	0	195	December 25, 2023
Generate raw word embeddings using transformer models like BERT for downstream process Beginners	9	38441	October 4, 2021
Choosing between monolingual and multilingual models Models	0	174	May 23, 2024
Training BERT for word embedding Beginners	17	13522	November 12, 2022

How to create word embeddings for non-English languages using BERT-like models?

Related topics