Hi all! I want to train T5 in a new language from scratch an I think the best way to do this is through the unsupervised denoising task. I’ve found that there is no function in huggingface to create the train data (masked data) as the T5 documentation indicates so I’ve tried to create by my own. A…

Train T5 from scratch

ArmanAsq April 26, 2024, 7:50am 5

Hi!
Any update on your code?
Is it possible to use for fine tune the T5 model?

Topic		Replies	Views
How to denoise text using T5? 🤗Transformers	2	692	May 8, 2023
Preprocessing for T5 Denoising Intermediate	1	2744	May 20, 2021
T5 masking - spans of text tokens or encoded tokens? Beginners	0	833	August 12, 2021
Prepare data for pretraining T5 model 🤗Datasets	1	1082	May 4, 2023
Question on HuggingFace's T5 documenation 🤗Transformers	0	321	May 18, 2023