from transformers import Trainer, TrainingArguments
training_args = TrainingArguments(
output_dir="/gdrive/MyDrive/Thesis/GPT2/checkpoints",
overwrite_output_dir=False,
num_train_epochs=5,
per_device_train_batch_size=6, #previous was 6
save_steps=100,
save_total_limit=5,
fp16 = True,
dataloader_drop_last=True,
#evaluate_during_training=True,
warmup_steps=200
)
trainer = Trainer(
model=model,
args=training_args,
data_collator=data_collator,
train_dataset=dataset,
# prediction_loss_only = True
)
trainer.train()
I want to save the checkpoints directly to my google drive. The problem is the code above saves my checkpoints upto to save limit all well. But after the limit it can’t delete or save any new checkpoints. Although it says checkpoints saved/deleted in the console. Any help?