2 GPUs don’t bring a lot of speedup compared to one since you add all those synchronization operations. The main speedup is that you should have double the batch size automatically so less iterations (unless you used max_steps in your command, as always, it’s hard to help you on what’s wrong without seeing the code your un).
1 Like