I would like to train a T5 model (t5-base version) without loading the pretrained weights, if I write the following:
from transformers import T5Config, T5Model
config = T5Config.from_pretrained(‘t5-base’)
model = T5Model(config)
It will produce the t5-base sized T5 model without loading the checkpoint weights?
Yes, that’s correct. You will definitely see the “pretrained weights” download progress bar in the other case
Thanks for your reply, if I will see the download progress bar it means that the pretrained weights have been loaded?
The fact that the T5Config was initialized by “from_pretrained” doesn’t imply downloading something?
That’s only config (eg. hyperparameters)
There will be a little progress bar of < 1mb. The weights are > 1GB.
Yeah, I’ve noticed it. Thank you all!