Train Tokenizer from scratch on Indic Lanuguages

vpkprasanna April 1, 2024, 11:20am 1

How to train a own tokenizer model from scratch on indic languages such as tamil , telugu , kannada and what are the steps to do it ?

Topic		Replies	Views
Train model from scratch on own dataset Beginners	0	489	February 26, 2024
Tamil ASR: Fine-Tuning Wav2Vec Languages at Hugging Face	2	646	April 1, 2021
Indian Languages - XLSR - Wav2Vec2 Fine- Tuning Languages at Hugging Face	0	921	March 29, 2021
How to build a multilingual tokenizer from scratch (for mbart)) Beginners	0	200	October 27, 2021
Is there any more tokenizer-free language model available? Models	0	551	March 12, 2022