Estimate of ALBERT and BERT's pre-training time and required hardware

yj2773 September 15, 2022, 3:11pm 1

Hi,
Can somebody please tell me what ALBERT and BERT’s pre-training time and required hardware are on a corpus of 100M sentences?

Topic		Replies	Views
Best Pre-training Strategy Research	0	715	March 3, 2022
Pre-trained models that weren't trained on Wikipedia? Intermediate	2	487	February 10, 2022
Albert MLM is slow 🤗Transformers	0	670	August 19, 2020
Are albert-base-v1( and v2) pretrained enough? 🤗Transformers	4	273	October 26, 2021
Cost to fine tune large transformer models on the cloud? Beginners	1	1186	November 29, 2021