Why is `accelerator.save` saving once for each node?

🤗Accelerate
1

Probably a dumb question: why is accelerator.save once for each node instead of once for all nodes? i.e. conditioned local_process_index==0 instead of process_index==0

see accelerate/other.py at cb54e1023e17763fa7f11d1de1307f85ba8b6c31 · huggingface/accelerate · GitHub :slight_smile:

2

Hello, I think it is done so that the user can load the model on any node later on. @sgugger and @muellerzr might have more to add on this.

1 Like
3

Exactly. We could add an option for multinode systems that share the same file storage though.

4 Likes