Training using FSDP, qLoRa on multinode

How to train a model like Llama3 using FSDP, qLoRa on two (or more) nodes if each node has one (or more) GPU?
Can anyone provide a link to a similar example?
I am grateful in advance for any help!

1 Like