Are there some ways to start fsdp2 using fsdp config or CLI arguments?
Currently, --fsdp
would initialize FullyShardedDataParallelPlugin(fsdp_version=1)
.
Trainer initializes Accelerator using the self-generated args, and we are now allowed to pass a FullyShardedDataParallelPlugin to Accelerator.
Any ideas?