Invalid username and password when push_to_hub set to true

Intermediate
1

During SFT fine-tuning, with push_to_hub set to true, push_to_hub_token specified, this error occurred

Generating table_gpt_no_think split: 100%|██████████| 13203/13203 [00:00<00:00, 198627.67 examples/s] Generating tulu_3_sft_personas_instruction_following_no_think split: 53%|█████▎ | 16000/29970 [00:00<00:00, 156104.8Generating tulu_3_sft_personas_instruction_following_no_think split: 100%|██████████| 29970/29970 [00:00<00:00, 164501.24 examples/s] Generating xlam_traces_no_think split: 100%|██████████| 59962/59962 [00:00<00:00, 409716.03 examples/s] Map: 100%|██████████| 2057/2057 [00:01<00:00, 1764.20 examples/s] Dataset sanity check passed for 5 examples. All seq_len=1024. Truncating train dataset: 100%|██████████| 2057/2057 [00:00<00:00, 86191.50 examples/s] Traceback (most recent call last): File "/root/.cache/uv/environments-v2/script-912247c0edd68a55/lib/python3.12/site-packages/huggingface_hub/utils/_http.py", line 407, in hf_raise_for_status response.raise_for_status() File "/root/.cache/uv/environments-v2/script-912247c0edd68a55/lib/python3.12/site-packages/requests/models.py", line 1026, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: ``https://huggingface.co/api/repos/create`` The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/tmp/script.py", line 342, in <module> trainer = SFTTrainer( ^^^^^^^^^^^ File “/root/.cache/uv/environments-v2/script-912247c0edd68a55/lib/python3.12/site-packages/trl/trainer/sft_trainer.py”, line 869, in _init_ super()._init_( File “/root/.cache/uv/environments-v2/script-912247c0edd68a55/lib/python3.12/site-packages/transformers/utils/deprecation.py”, line 172, in wrapped_func return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File “/root/.cache/uv/environments-v2/script-912247c0edd68a55/lib/python3.12/site-packages/transformers/trainer.py”, line 691, in _init_ self.init_hf_repo() File “/root/.cache/uv/environments-v2/script-912247c0edd68a55/lib/python3.12/site-packages/transformers/trainer.py”, line 4964, in init_hf_repo repo_url = create_repo(repo_name, token=token, private=self.args.hub_private_repo, exist_ok=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File “/root/.cache/uv/environments-v2/script-912247c0edd68a55/lib/python3.12/site-packages/huggingface_hub/utils/_validators.py”, line 114, in _inner_fn return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File “/root/.cache/uv/environments-v2/script-912247c0edd68a55/lib/python3.12/site-packages/huggingface_hub/hf_api.py”, line 3766, in create_repo hf_raise_for_status(r) File “/root/.cache/uv/environments-v2/script-912247c0edd68a55/lib/python3.12/site-packages/huggingface_hub/utils/_http.py”, line 480, in hf_raise_for_status raise _format(HfHubHTTPError, str(e), response) from e huggingface_hub.errors.HfHubHTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/api/repos/create (Request ID: Root=1-68e5c777-452112025d4acbd549ec978f;775ceb4a-389e-4816-80be-0e3c65330995)

Invalid username or password.

Please what might be wrong?

1 Like
2

The argument name might be wrong. Also, for programs you’re using yourself, I think it’s easier to use the huggingface_hub library’s login function or the HF_TOKEN environment variable than passing token each time.

from trl import SFTConfig  # or SFTTrainer(..., args=TrainingArguments(...))
args = SFTConfig(
    push_to_hub=True,
    hub_model_id="your-username/your-repo",
    #push_to_hub_token=os.environ["HF_TOKEN"],   # <- wrong
    hub_token=os.environ["HF_TOKEN"],   # <- correct
    hub_private_repo=True,
)
3

both ``push_to_hub_token`and``hub_token`` were specified and the value passed was confirmed as the same token string used for ```hf auth login```

1 Like
4

Hmm… Try isolating problem… like this:

#   pip install -U huggingface_hub
import os, json, subprocess
from huggingface_hub import login, whoami, HfApi

os.environ["HF_TOKEN"] = "hf_xxx"
os.environ["TARGET_REPO"] = "you/debug-repo"

t = os.getenv("HF_TOKEN")
if t:
    login(token=t, add_to_git_credential=False, skip_if_logged_in=True)

w = whoami(); print("whoami:", json.dumps(w))
repo = os.getenv("TARGET_REPO", f"{w.get('name','user')}/hf-push-probe")
api = HfApi()
try:
    api.create_repo(repo_id=repo, repo_type="model", private=True, exist_ok=True, token=t)
    print("API:create_repo:ok")
except Exception as e:
    print("API:create_repo:error", e)

env = dict(os.environ); env["GIT_TERMINAL_PROMPT"] = "0"
subprocess.run(["git","lfs","version"], check=False)
cred = subprocess.run(["git","credential","fill"], input="url=https://huggingface.co\n",
                      text=True, capture_output=True).stdout
print("GIT:cred", "present" if "password=" in cred else "absent")
rc = subprocess.run(["git","ls-remote", f"https://huggingface.co/{repo}"], env=env).returncode
print("GIT:ls-remote", "ok" if rc == 0 else "fail")
5

even with that, the git username/password popped up but failed to auth as it is deprecated

gitpopup

popup
popup416×309 5.56 KB

1 Like
6

When using Git on Windows, you’ll likely run into errors unless you manually install Git LFS and Git beforehand. Whether that’s the cause of this particular issue is another matter…

7

Oh yes, thanks. The issue was that using username/password auth for git is no longer supported so even if token is already setup for the project, it is still required to manually add the token to the project repo that resides locally

Deprecation of Git Authentication using password

``
$: git remote set-url origin ``https://<user_name>:<token>@huggingface.co/``<repo_path> $: git pull origin ```

Typically, using ``hf auth login`` should ask whether to add the token as git credential, but as the directory running the script did not have git initialized locally, the option was bypassed. But looks better now.

1 Like
8

but the issue persists :pensive_face:

1 Like
9

do you think the repo (space) should be created first, before running the job, with push to hub set to ``True``?

1 Like
10

Probably it won’t have any effect. If not, I think the repository will likely be created automatically…

11

There shouldn’t be many possibilities left…

12

something is wrong!

Have you been able to push to hub?

The issue persisted even after specifying the ```secrets`option as`hf jobs uv run –secrets hf_token``

1 Like
13

Hmm… I don’t really use push_to_hub much myself. I mostly upload using upload_folder.

Come to think of it, I remember that the implementation of push_to_hub varies across libraries—sometimes they’re the same, sometimes they’re different. (Depends on the version)

Also, I think the latest HF libraries probably prioritize XET over LFS. (If it’s installed: pip install -U huggingface_hub[hf_xet])

Edit:
BTW, HF_TOKEN must be uppercase? hf jobs uv run --secrets HF_TOKEN

14

The UAT was passed directly into the CLI.

1 Like
15

Hmm… Try debugging…

# Minimal HF Git auth probe
# 1) Passwords for Git are disabled → use PAT or SSH
#    https://huggingface.co/blog/password-git-deprecation
# 2) Login and store PAT in Git helper
#    https://huggingface.co/docs/huggingface_hub/en/package_reference/authentication
# 3) HF_TOKEN env is the canonical variable
#    https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables
# 4) Inspect Git's credential helper behavior
#    https://git-scm.com/docs/git-credential

import os, json, subprocess, sys, re
from huggingface_hub import login, whoami, HfApi

def sh(cmd, **kw):
    p = subprocess.run(cmd, text=True, capture_output=True, **kw)
    return p.returncode, (p.stdout or "") + (p.stderr or "")

tok = os.getenv("HF_TOKEN")
if not tok:
    print("Set HF_TOKEN"); sys.exit(1)

# Save PAT to Git helper so HTTPS push does not prompt  (2)
login(token=tok, add_to_git_credential=True, skip_if_logged_in=True)

# Prove token works for HTTP API  (2)(3)
me = whoami()
print("whoami:", json.dumps(me))

# Create or confirm a repo to test permissions  (2)
rid = os.getenv("TARGET_REPO", f"{me.get('name','me')}/hf-push-probe")
HfApi().create_repo(repo_id=rid, private=True, exist_ok=True, token=tok)

# Remote sanity: accept HTTPS or SSH pattern
rc, remote = sh(["git","config","--get","remote.origin.url"])
if rc == 0:
    ok = re.match(r"^https://huggingface\.co/[^/]+/[^/]+(?:\.git)?$", remote) or \
         re.match(r"^git@hf\.co:[^/]+/[^/]+\.git$", remote)
    print("origin:", remote, "OK" if ok else "MALFORMED")

# Git helper must return a password= (the PAT) for huggingface.co  (4)
rc, cred = sh(["git","credential","fill"], input="url=https://huggingface.co\n\n")
print("git credential fill has password:", "password=" in cred)

# Non-interactive HTTPS check. Should be OK if PAT is saved.  (1)(4)
env = dict(os.environ); env["GIT_TERMINAL_PROMPT"] = "0"
rc, out = sh(["git","ls-remote", f"https://huggingface.co/{rid}"], env=env)
print("ls-remote:", "OK" if rc == 0 else f"FAIL\n{out}")