Hi, I want to ask how to get the grad norm of a prepared model especially with DeepSpeed-Zero3. The model.parameters()
is empty.
Thanks!
Hi, I want to ask how to get the grad norm of a prepared model especially with DeepSpeed-Zero3. The model.parameters()
is empty.
Thanks!
same question. I also want to assess the grad_norm for each parameter matrix. but i got None.
There seem to be two ways to obtain this: by using an utility method or by getting manually from DeepSpeed.