`num_processes == 1` even when I set it to `--num_processes 2`

I have a Python script that I’m running with HuggingFace Accelerate. I also have a total of 4 GPUs but only wish to use 2. The command I’m running looks like this: accelerate launch --num_processes 2 script.py. However, whenever I run it this way, it says that the num_processes is 1. It seems as of now that I can only use either all of the GPUs or 1. Is there anything that I’m doing wrong or should look into further?

Try passing in --multi_gpu as well

I tried that and it didn’t work. It also wouldn’t make sense though, would it? Not passing any argument makes the process grab all of the GPUs by default, so I don’t understand how specifying the number of processes would change that.

Can you provide the output of nvidia-smi, and the full output of just doing accelerate launch myscript.py?

Funny, now when I tried passing --multi_gpu as well again, it seems to be working. Not sure what went wrong before but I assume this is how you do it.

Do you know if there’s a reason why --multi_gpu is not set to True by default?

Nope, going to be looking into enabling it :slight_smile: