There are probably many unknown bugs related to multi-GPU, so it may be quicker to investigate the cause of the error and resolve each case individually. In some cases, the issue may be resolved by simplifying the operation, such as setting device_map=“sequential”.