Hey @darrenoakey,
You may want to take a look at this answer of mine to see how to load the model (fully or partially) on CPU: How to use trust_remote_code=True with load_checkpoint_and_dispatch? - #2 by abhinavkulkarni
Hey @darrenoakey,
You may want to take a look at this answer of mine to see how to load the model (fully or partially) on CPU: How to use trust_remote_code=True with load_checkpoint_and_dispatch? - #2 by abhinavkulkarni