How to create word embeddings for non-English languages using BERT-like models?

ekdnam · March 22, 2021, 7:05pm

Hi all! This is my first topic here, so apologies in case I make some errors.

Currently I am working on creating custom word embeddings for an Indian language, Marathi. They will be later used for creating a NMT model for translation between Marathi and English.

How to do so using transformers? Also, what is the required data cleaning process?

Topic		Replies	Views
Creating word embeddings using BERT of machine generated sequential data Models	0	201	April 7, 2023
Using custom embeddings for pre-training model for new vocabulary Beginners	0	122	December 25, 2023
Generate raw word embeddings using transformer models like BERT for downstream process Beginners	9	34806	October 4, 2021
Training BERT for word embedding Beginners	17	10402	November 12, 2022
fine tuning encoder decoder for custom language translation Models	0	417	April 22, 2021

How to create word embeddings for non-English languages using BERT-like models?

Related Topics