Hey @aclifton314 , you can use “grouping” in wandb to achieve what you’re trying to do I think.
One way is to pass the “group” argument to wandb.init with your own name or id for the experiment, e.g. “my_experiment_1”. Then you will be able to group runs from the same run into one in the UI, e.g. your plots can show the average loss instead of the loss for the 3 individual GPUs used in “my_experiment_1”
To pass arguments to wandb.init using Accelerate, you can use the init_kwargs keyword, and pass it a nested dictionary, in the below case, this passes an argument to the group argument in wandb.init
accelerator.init_trackers("my_project", config=hps, init_kwargs={"wandb":{"group":"my_experiment_1"}})