Hello :slight_smile: I’m using deepspeed to train big models on two GPUs (that cannot fit on one). I’m using the hugging face Trainer with deepspeed params like so : trainer = Trainer( model=model, train_dataset=CustomDataset(...), eval_dataset=CustomDataset(...), tokenizer=token…

[Solved] Cannot restart training from deepspeed checkpoint

HereWe December 28, 2023, 3:41am 4

It works for me!

Topic		Replies	Views
Checkpoint breaks with deepspeed 🤗Transformers	6	3458	March 20, 2021
Deepspeed and Trainer does not exit after training is completed Beginners	1	215	July 30, 2024
Load a single GPU checkpoint to 2 GPUS (deepspeed) Intermediate	0	2028	June 29, 2022
Huggingface --resume_from_checkpoint feature with deepspeed Beginners	0	520	November 11, 2021
Deepspeed resume training from saved states 🤗Accelerate	0	1283	September 8, 2022