Connecting to hub account from Databricks

I’m running transformers in a Databricks notebook with a local dataset. the task is text classification with BERT and DistilBERT. I have no problem loading public checkpoints from the hub and fine-tuning. The problem comes when I want to push the model back to my account on the hub. The cell

from huggingface_hub import notebook_login
notebook_login()

doesn’t allow me to log in. Figuring I need to display HTML I tried

displayHTML(notebook_login())

but this gives me a java NullPointerException. Has anybody succeeded in connecting to their account from a DataBricks notebook?

thanks

Hi Alun!

Today I ran into the same challenge. Hugging Face unfortunately seems to lack proper documentation on how to login into the Hugging Face Hub from within a Databricks notebook. While I was unsuccessful in logging in through the currently documented huggingface-cli login, notebook_login() and HfApi.set_access_token(), I was successful in logging into and pushing models and datasets to the Hugging Face Hub through a hacky implementation of what is supposed to be or become a deprecated method.

Start with installing Hugging Face Hub through:
%pip install huggingface_hub

And installing git-lfs through:

%sh
# https://stackoverflow.com/questions/48734119/git-lfs-is-not-a-git-command-unclear
curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
sudo apt-get install git-lfs

Then, continue with logging in through:

from huggingface_hub.commands.user import _login
from huggingface_hub import HfApi

api = HfApi()
_login(hf_api = api, username = "USER", password = "PASS")

And:

%sh
git config --global credential.helper store
# https://github.com/huggingface/notebooks/blob/main/examples/language_modeling_from_scratch-tf.ipynb
git config --global user.email "EMAIL"
git config --global user.name "FULLNAME"

This worked for me and took quite some trial and error to figure out. Pushing models still behaves a bit weird due to some occasionally occuring git errors.

I hope this helps!

It seems _login() has seen been updated. You can still follow the login procedure above for the most part, but you need to use a token instead of username/password combination with _login():

from huggingface_hub.commands.user import _login
from huggingface_hub import HfApi

api = HfApi()
_login(hf_api = api, token = "TOKEN")

Tokens can be generated in the Access Tokens page of your Hugging Face profile.