torch.Size([0]) on some layers when using Accelerate

When using Accelerate, I am running into an error where some of my model layers have a size of 0 when saving the model. This leads to a mismatch error when I try to load the fine-tuned model later on. The issue doesn’t seem to happen when I run the script without accelerate launch. Any help with this would be much appreciated.

Thanks in advance!

Did you solve this question? I have the same problem

Hi @daiYJ, can you provide a reproducer ?