Where to put use_auth_token in the code if you can't run hugginface-cli login command?

Beginners
1

I am training a bart_large_cnn model for summarization, where I have used : -

training_args = Seq2SeqTrainingArguments(
    output_dir="results",
    num_train_epochs=1,  # demo
    do_train=True,
    do_eval=True,
    per_device_train_batch_size=4,  # demo
    per_device_eval_batch_size=4,
    # learning_rate=3e-05,
    warmup_steps=500,
    weight_decay=0.1,
    label_smoothing_factor=0.1,
    predict_with_generate=True,
    logging_dir="logs",
    logging_steps=50,
    save_total_limit=3,
    push_to_hub=True,
)

but when I try to compile :-

trainer = Seq2SeqTrainer(
    model=model,
    args=training_args,
    data_collator=data_collator,
    train_dataset=train_data,
    eval_dataset=validation_data,
    tokenizer=tokenizer,
    compute_metrics=compute_metrics
)

it gives an error :-

HTTPError: Invalid user token. If you didn't pass a user token, make sure you are properly logged in by executing huggingface-cli login, and if you did pass a user token, double-check it's correct.

Now, in the user documentation, its written that either pass the token as :-

!huggingface-cli login

or

use_auth_token='token_value'

I tried putting this token value as below :-

the first command (cli-login) doesn’t run (takes forever). so I used the second option as below; -

model = AutoModelForSeq2SeqLM.from_pretrained(model_name,use_auth_token='token_value')

tokenizer = AutoTokenizer.from_pretrained(model_name,use_auth_token = 'token_value')

but the same error keeps popping up.
Where else can I put this token value to push my model to the hub?

1 Like
2

I added the following parameter (hub_token) in the training_args as below :-

training_args = Seq2SeqTrainingArguments(
    output_dir="results",
    num_train_epochs=1,  # demo
    do_train=True,
    do_eval=True,
    per_device_train_batch_size=4,  # demo
    per_device_eval_batch_size=4,
    # learning_rate=3e-05,
    warmup_steps=500,
    weight_decay=0.1,
    label_smoothing_factor=0.1,
    predict_with_generate=True,
    logging_dir="logs",
    logging_steps=50,
    save_total_limit=3,
    push_to_hub=True,
    hub_token = 'token value'
)

but when I try to compile :-

trainer = Seq2SeqTrainer(
    model=model,
    args=training_args,
    data_collator=data_collator,
    train_dataset=train_data,
    eval_dataset=validation_data,
    tokenizer=tokenizer,
    compute_metrics=compute_metrics
)

FileNotFoundError: [WinError 2] The system cannot find the file specified
During handling of the above exception, another exception occurred:
File "C:\Users\abhay.saini\AppData\Local\Programs\Python\Python39\lib\site-packages\huggingface_hub\repository.py", line 489, in check_git_versions
    raise EnvironmentError(
OSError: Looks like you do not have git-lfs installed, please install. You can install from https://git-lfs.github.com/. Then run `git lfs install` (you only have to do this once).

I did then try installing git-lfs

pip install git-lfs
Requirement already satisfied: git-lfs in c:\users\abhay.saini\appdata\local\programs\python\python39\lib\site-packages (1.6)

but the error persists…
Can anyone help?

3

You need to do the git lfs install command for your last problem. As you found out, the hub_token argument is the right way to pass along your token!

4

getting same error, did you manage to solve this?

2 Likes
5

accept this

6

I am getting the same error even after using pip install git-lfs. Anyone managed to find a solution?