How to get the grad norm of a deepspeed-zero3 model after accelerator.prepare()

Hi, I want to ask how to get the grad norm of a prepared model especially with DeepSpeed-Zero3. The model.parameters() is empty.

Thanks!

1 Like

same question. I also want to assess the grad_norm for each parameter matrix. but i got None.

1 Like

There seem to be two ways to obtain this: by using an utility method or by getting manually from DeepSpeed.