How to run 30B meta model on two nodes with accelerate?

🤗Accelerate
#1

Hello,

I can successfully run the 30B meta model on one node (following load_checkpoint_and_dispatch "Expected all tensors to be on the same device" for > 1 GPU devices · Issue #362 · huggingface/accelerate · GitHub). Now I was curious if I can run the same on two nodes to prepare for even larger models. I ran “accelerate config” and “accelerate launch my_script.py” on both nodes, but it seems that the model is just completely loaded on each of the two nodes.

#2

There is nothing to dispatch the model between nodes right now (and it’s probably too complicated to be added soon).

#3

ok, that’s what I feared. Thank you very much.