BERT model trained on small corpus (English)?

macabdul9 May 8, 2021, 10:06pm 1

Is there any transformer-based model pre-trained on very small data ( less than BERT)?

Topic		Replies	Views
Using EXTREMELY small dataset to finetune BERT 🤗Transformers	6	8831	February 1, 2023
Further pre-train language model in transformers like BERT Models	3	1008	March 27, 2022
Pre-Train BERT (from scratch) Research	43	17449	June 27, 2022
Any BERT model recommendation needed for getting feature of structured sentences Beginners	0	295	June 8, 2022
Best Pre-training Strategy Research	0	715	March 3, 2022