There seems to be an internal authentication issue when deploying a private LoRA adapter with Inference Endpoints / TGI 3.1.0: We have a private LoRA of meta-llama/Llama-3.2-3B-Instruct on our organization account, and I’m trying to deploy it on an Inference Endpoint serving this Llama base model, …

Hi @nikos-ir Can you please also add your hf_token as an env variable to the Endpoint and try once more? Let us know if you’re still running into issues.

Inference Endpoint can't access private LoRA

Inference Endpoints on the Hub

meganariley February 27, 2025, 4:44pm 2

Hi @nikos-ir We’re taking a look into this and I’ll update you soon.

2 Likes

Topic		Replies	Views
Custom handler with gated model Inference Endpoints on the Hub	5	827	January 25, 2024
Cannot Deploy My Private Model for Inference Endpoint Inference Endpoints on the Hub	6	1062	April 24, 2023
Error Deploying Private Endpoint Inference Endpoints on the Hub	2	304	October 23, 2023
Problem to deploy endpoint Inference Endpoints on the Hub	3	304	July 19, 2024
Inference endpoint, gated repo 401 error Inference Endpoints on the Hub	4	206	July 25, 2024

Inference Endpoint can't access private LoRA

Related topics