BERT model trained on small corpus (English)?

macabdul9 May 8, 2021, 10:06pm 1

Is there any transformer-based model pre-trained on very small data ( less than BERT)?

Topic		Replies	Views
Using EXTREMELY small dataset to finetune BERT 🤗Transformers	6	12734	February 1, 2023
Transformers for small datasets? Beginners	3	57	October 9, 2024
Further pre-train language model in transformers like BERT Models	3	1100	March 27, 2022
Pre-Train BERT (from scratch) Research	43	18886	June 27, 2022
Any BERT model recommendation needed for getting feature of structured sentences Beginners	0	393	June 8, 2022