Is there any codebase in huggingface that could be used to pretrain T5 model? Looking into the examples dir in the repo there is nothing mentioned about T5. Thanks!

Example of how to pretrain T5?

jessicalopez October 17, 2022, 3:47pm 15

Hello @valhalla , do you have any updates? Thank you in advance

Topic		Replies	Views
Pre-training googlebyt5small 🤗Transformers	0	236	October 26, 2022
Training T5 on mlm task from scratch 🤗Transformers	4	3298	July 29, 2022
Prepare data to fine-tune T5 model on unsupervised objective 🤗Transformers	2	3952	November 3, 2021
Prepare data for pretraining T5 model 🤗Datasets	1	1093	May 4, 2023
How is T5 pretrained? 🤗Transformers	3	523	July 12, 2021