How to create word embeddings for non-English languages using BERT-like models?

ekdnam · March 22, 2021, 7:05pm

Hi all! This is my first topic here, so apologies in case I make some errors.

Currently I am working on creating custom word embeddings for an Indian language, Marathi. They will be later used for creating a NMT model for translation between Marathi and English.

How to do so using transformers? Also, what is the required data cleaning process?

Topic		Replies	Views
Creating word embeddings using BERT of machine generated sequential data Models	0	254	April 7, 2023
Using custom embeddings for pre-training model for new vocabulary Beginners	0	192	December 25, 2023
Generate raw word embeddings using transformer models like BERT for downstream process Beginners	9	37958	October 4, 2021
Choosing between monolingual and multilingual models Models	0	167	May 23, 2024
Training BERT for word embedding Beginners	17	13163	November 12, 2022

How to create word embeddings for non-English languages using BERT-like models?

Related Topics