Hi all, I’m relatively new to Huggingface, Transformers and especially Accelerate. I’m trying to fine-tune to the CodeGen model using four GPUs, distributing the training across each GPU to speed up compute and prevent running out of CUDA memory. Note that I don’t want to replicate the model on eac…

import pickle as pkl and use the pickle.dump method eg: [Peterstone] Saving and loading objects and using pickle python, object, pickle asked by Peterstone on 03:17PM - 25 Dec 10 UTC pickle.dump() you may find that all but …

Can't pickle error using accelerate multi-GPU

🤗Accelerate

Johnguitarist March 7, 2023, 11:12am 7

import pickle as pkl
and use the pickle.dump method eg:

pickle.dump()

you may find that all but one of your objects can be pickled, which will be the culprit

Topic		Replies	Views
How can I get advantage using multi-GPUs Beginners	5	3122	February 3, 2021
Using `torch.distributed.all_gather_object` returns error when using 1 GPU but works fine for multiple GPUs 🤗Accelerate	3	2806	July 5, 2023
Error when fine-tuning on multi-gpu 🤗Transformers	1	172	February 17, 2025
Multiple gpu not properly parallelized during model.generate() 🤗Transformers	4	1581	October 9, 2022
Having trouble accelerate on my 2 GPU machine Beginners	0	717	May 24, 2023

Can't pickle error using accelerate multi-GPU

Related topics