Does the REST API work with private repo?

🤗Datasets
1

I was experimenting with the REST API with a private repo. Despite providing the user access token in the request header, I receive an error

import requests
from dotenv import load_dotenv
load_dotenv()
per_token = os.getenv('API_PER_TOKEN')
headers = {"Authorization": f"Bearer {per_token}"}
API_URL = "https://datasets-server.huggingface.co/is-valid?dataset=sl02/np-datasets"
def query():
    response = requests.request("GET", API_URL, headers=headers)
    return response.json()
data = query()

{'error': 'The dataset does not exist, or is not accessible without authentication (private or gated). Please retry with authentication.'}
However, when I make the repository public, it returns {'valid': True}. But, when I run the first-rows API, I get the following message

import requests
from dotenv import load_dotenv
load_dotenv()
per_token = os.getenv('API_PER_TOKEN')
headers = {"Authorization": f"Bearer {per_token}"}
API_URL = "https://datasets-server.huggingface.co/first-rows?dataset=sl02/np-datasets&config=default&split=train"
def query():
    response = requests.request("GET", API_URL)
    return response.json()
data = query()

{'error': 'The response is not ready yet. Please retry later.'}

The load_dataset() works in private mode when I set the use_auth_token argument. Any clue what I missing here?

1 Like
2

Maybe @severo knows more, but IIRC the REST API is not available yet for private repos.

3

Hi @sl02. The REST API uses the same rule as the dataset viewer (see The Dataset Preview has been disabled on this dataset - #6 by severo): it’s not available at all for the private datasets for now.

re “The response is not ready yet. Please retry later”: the responses to the API endpoints are pre-computed asynchronously and can take some time to be processed, depending on the dataset itself and on the load of the servers.