Prompt Tuning for casual LM

I am trying to do prompt tuning for casual LM on custom dataset on Bloom560m.

While I am not getting any error, during training, epoc loss and tensors is showing Nan.

what could be the reason and how to resolve?