Is there any documentation on how to use prepare in various cases? I have two models but want to use the same optimizer. How should I use the prepare()
in such a case?
I’m training the two models in tandem, say model1
and model2
. When calling the prepare()
function, can I call it separately as in the below code?
model1, dataloader, optimizer = accelerator.prepare(model1, dataloader, optimizer)
if some_condition:
model2 = acceletator.prepare(model2)
Or do I have to call prepare()
only once, as in the below code?
if not some_condition:
model1, dataloader, optimizer = accelerator.prepare(model1, dataloader, optimizer)
else:
model1, model2, dataloader, optimizer = acceletator.prepare(model1, model2, dataloader, optimizer)
I’ve asked this on stackoverflow as well