How to get the grad norm of a deepspeed-zero3 model after accelerator.prepare()

[Hannibal046]

Hi, I want to ask how to get the grad norm of a prepared model especially with DeepSpeed-Zero3. The model.parameters() is empty.

Thanks!

[weeknan]

same question. I also want to assess the grad_norm for each parameter matrix. but i got None.

[John6666]

There seem to be two ways to obtain this: by using an utility method or by getting manually from DeepSpeed.