Is there an easy way to obtain the sharded-fp16 version of a model after having loaded it with from_pretrained?
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Loading a peft model which is saved on multiple nodes using sharded_state_dict? | 0 | 49 | August 2, 2024 | |
| Should I shard dataset in distributed training? | 2 | 719 | December 3, 2021 | |
| Loading a model which is saved on multiple nodes using sharded_state_dict? | 0 | 92 | August 13, 2024 | |
| How does this work? (Downloading multi-part models) | 0 | 2056 | May 20, 2023 | |
| FSDP accelerate.prepare gives OOM. How to load model into single GPU, then distribute shards? | 2 | 1197 | January 24, 2024 |