BERT model trained on small corpus (English)?

macabdul9 May 8, 2021, 10:06pm 1

Is there any transformer-based model pre-trained on very small data ( less than BERT)?

Topic		Replies	Views
Using EXTREMELY small dataset to finetune BERT 🤗Transformers	6	13144	February 1, 2023
Transformers for small datasets? Beginners	3	76	October 9, 2024
Further pre-train language model in transformers like BERT Models	3	1108	March 27, 2022
Pre-Train BERT (from scratch) Research	43	18993	June 27, 2022
Any BERT model recommendation needed for getting feature of structured sentences Beginners	0	400	June 8, 2022