Using loaded model with accelerate for inference

Why are using load_checkpoint_and_dispatch when there is nothing to dispatch (e.g. no device_map)? You can just use load_checkpoint_in_model :slight_smile:

1 Like