Resuming Training from Checkpoints Stored on Hugging Face Hub (without downloading manually)

[John6666]

I think the Trainer’s original method and the method using from_pretrained, which is often used even when simply using a model, are simple.

Continuing Pre Training from Model Checkpoint

Hi, I pre-trained a language model for my own data and I want to continue the pre-training for additional steps using the last checkpoint. I am planning to use the code below to continue the pre-training but want to be sure that everything is correct before starting. Let’s say that I saved all of my files into CRoBERTa. model = RobertaForMaskedLM.from_pretrained(‘CRoBERTa/checkpoint-…’) tokenizer = RobertaTokenizerFast.from_pretrained(‘CRoBERTa’, max_len = 512, padding = ‘longest’) training…

What is the purpose of save_pretrained()?

Hello everyone. I hope my question is not too silly, but there is something that confuses me. Let’s say I load a huggingface model using from_pretrained() method, and then finetune it using the Trainer class. Now, via TrainingArguments, I get the chance to define an argument called output_dir. If I specify a directory here, won’t my model be saved in this directory, thus enabling me to load it again in the future from this folder, using from_pretrained()? Here lies my question: if this argumen…