Issue while loading file-tuned gemma2

Models
4

I resolved it by following below steps:-

  1. push adapter weights(from trainer)
    trainer.push_to_hub(“your_huggingface_dir”)

  2. push the base model as well
    base_model.push_to_hub(“your_huggingface_dir”)

  3. then loading the model with:
    model_finetuned = AutoModelForCausalLM.from_pretrained(
    finetuned_model_name,
    device_map=“auto”,
    torch_dtype=torch.bfloat16
    )

1 Like