While analysing the GPU memory footprint of a few models ‘prepared’ via accelerator.prepare(), I found the models occupying more than twice the GPU memory compared to normally loaded models on device. What can be the reason for this inflated memory footprint? I am using a single V100 32 GB GPU and …

Inflated GPU memory footprint of model prepared via accelerate

varadhbhatnagar September 15, 2023, 4:40am 5

1 Like

Topic		Replies	Views
Training memory footprint depends on instantiating method Models	1	43	October 23, 2024
`Accelerator.prepare` utilize only one GPU instead of all the 8 available GPUs and raises "CUDA out of memory" 🤗Accelerate	3	2792	July 19, 2024
Data Parallel Multi GPU Inference 🤗Accelerate	9	4480	September 15, 2023
Accelerator OOM 🤗Accelerate	2	1202	July 5, 2023
Loading model directly to GPU omitting RAM Beginners	6	35	March 28, 2025