Problem spanning an endpoint from a personal repository with a custome handler

Inference Endpoints on the Hub
1

I am trying to span an inference endpoint for Falcon 40B.
However I want to have a custom handling of the model and its output.
So I duplicated the repo into my account and created the handler.py file.

However before even the logs start being generated, I am getting an error as following:

Exit code: 1. Reason: 1487, in _hf_hub_download_to_local_dir
    _download_to_tmp_and_move(
  File "/usr/local/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1884, in _download_to_tmp_and_move
    http_get(
  File "/usr/local/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 459, in http_get
    r = _request_wrapper(
        ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 396, in _request_wrapper
    hf_raise_for_status(response)
  File "/usr/local/lib/python3.11/site-packages/huggingface_hub/utils/_errors.py", line 367, in hf_raise_for_status
    raise HfHubHTTPError(message, response=response) from e
huggingface_hub.utils._errors.HfHubHTTPError: 

403 Forbidden: None.
Cannot access content at: https://cdn-lfs.hf.co/repos/bf/e9/...
If you are trying to create or update content,make sure you have a token with the `write` role.
Last status 1
Unable to download repo correctly after 6 attempts

Any though why this might be happening as spanning Falcon 40B itself worked out.
N.B: The repo is owned by my account.

1 Like
2

I think this happens when the README.md (YAML part) is improperly set up, or when the private setting is used but the token is not passed. That model doesn’t seem to be gated.

4

Thanks for sharing thoughs :-).

How you can pass the token when you are on inference endpoint interface.
set HF_TOKEN as an environment variable? This didn’t work.
Also the repo is mine personal public and it is not gated.
Also the repo is duplicated from the original Falcon 40B so the readme should be valid.

5

The private repo will be in 404 status for others or even for the owner without a password, so I think we need to give them a token or make it public somehow.
As for how to pass a token, you can pass it to almost all HF function with the token= argument.

6

I am talking in the setting of an “Inference Endpoint”. You can’t pass a token programmatically. Again the repo is public and you can download it normally through colab.

7

I am talking in the setting of an “Inference Endpoint”.

Very sorry… I was completely wrong. Surely here is the Endpoint category!
If so, it’s a strange error?