TRL training script authentication problem

Beginners
1

I have access to gemma-7b. I’m trying to fine-tune it with TRL following the instructions here: Welcome Gemma - Google’s new open LLM

In particular, I’m using the following script:

accelerate launch --config_file examples/accelerate_configs/multi_gpu.yaml --num_processes=1
examples/scripts/sft.py
–model_name google/gemma-7b
–dataset_name OpenAssistant/oasst_top1_2023-08-25
–per_device_train_batch_size 2
–gradient_accumulation_steps 1
–learning_rate 2e-4
–save_steps 20_000
–use_peft
–lora_r 16 --lora_alpha 32
–lora_target_modules q_proj k_proj v_proj o_proj
–load_in_4bit
–output_dir gemma-finetuned-openassistant

Unfortunately this throws me the following error:

Cannot access gated repo for url https://huggingface.co/google/gemma-7b/resolve/main/config.json.
Repo model google/gemma-7b is gated. You must be authenticated to access it. - silently ignoring the lookup for the file config.json in google/gemma-7b.
warnings.warn(
/home/…/miniconda3/envs/gemmamodelenvironment/lib/python3.10/site-packages/peft/utils/save_and_load.py:154: UserWarning: Could not find a config file in google/gemma-7b - will assume that the vocabulary was not modified.
warnings.warn(

It seems like the authentication in question should be done by passing my User Access Token for the model to the example script. Is that correct? If so, what is the named argument for this token in the example script? If I’m mistaken about the source of the error, what do I need to do to get this example script to run without errors?

2

Can you instead try running:

huggingface-cli login

and then passing your access token?

3

Hello,

Thanks very much for your response! It looks like your suggestion did the trick. It still produced some warnings but nothing fatal. I guess I’ll find out if it really worked when I try generating with it.

4

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.