How to start fsdp2 when using trainer?

Are there some ways to start fsdp2 using fsdp config or CLI arguments?

Currently, --fsdp would initialize FullyShardedDataParallelPlugin(fsdp_version=1).

Trainer initializes Accelerator using the self-generated args, and we are now allowed to pass a FullyShardedDataParallelPlugin to Accelerator.

Any ideas?

1 Like