Following Train a diffusion model, it illustrates that we can set the number of GPUs (default 1):
Phew, that was quite a bit of code! But you’re finally ready to launch the training with
Accelerate’s notebook_launcher function. Pass the function the training loop, all the training arguments, and the number of processes (you can change this value to the number of GPUs available to you) to use for training:
from accelerate import notebook_launcher args = (config, model, noise_scheduler, optimizer, train_dataloader, lr_scheduler) notebook_launcher(train_loop, args, num_processes=1)
Yet, when I use a VastAI instance with two GPUs, and pass 2 to the function, following error happens:
ValueError: To launch a multi-GPU training from your notebook, the `Accelerator` should only be initialized inside your training function. Restart your notebook and make sure no cells initializes an `Accelerator`.