Training using multiple GPUs

Since you are not providing the code of that model, I can’t really help you with what is going wrong.