Quoting from here
If you’re migrating from ZeRO-2 configuration note that
allgather_partitions
,allgather_bucket_size
andreduce_scatter
configuration parameters are not used in ZeRO-3. If you keep these in the config file they will just be ignored.
How did you know? Because when I read documentation of deepspeed configuration. There is never any mention about which parameters will be ignored under ZeRO-3.