n00b question - I’ve been trying to deploy Llama 3 from Hugging Face on Google’s Vertex AI (from the Model Garden) but I can’t do it as I keep getting an error with my access token. Google keeps telling me that ‘The token is not valid or not have permission’.
I’ve tried read only tokens, write tokens, fine-grained tokens with all permissions enabled and just can’t get it to work.
Anyone have any pointers?
Thanks
Hi,
I’ve deployed LLaMa myself on Vertex AI and it worked.
Do note that if you set a token at Hugging Face – The AI community building the future. you need to safely store it and use it, cause everytime you refresh the token the previous one no longer works. A read-only token should work fine.
Thanks I’m aware of this and have made sure that I’m trying the new token every time I refresh it.
This issue was resolved for me when creating a new HF token after gaining access to the Llama 3 model. Refreshing old tokens did not work. Hope this works for you, I have been having the same issue when deploying from HF to Google Vertex.
Verifying Vertex AI permissions can be tricky. Have you double-checked the service account you’re using has the Vertex AI Service Agent role? That might be the culprit. Also, ensure that any related resources, like Burlington handles, are correctly configured and accessible.
Thanks - I tried this but unfortunately it still doesn’t work
Thanks - I added that role to my service account, but still have the same issue
I have been able to create a specific token for Vertex AI, and added the permission to access the repo of the model I am trying to use with Model Garden (codestral). It is accepted by GCP/Vertex and the download+deployment process starts.
But several minutes later, I have a download error from GCP…
Hi @seanbetts check for few tips:
I hope this helps 
1.Ensure your access token is fine-grained and grants “Read access to contents of all public gated repos you can access.”
2.Accept the license for the chosen model in your Hugging Face account and acknowledge the license agreement.
Once these two steps are completed, your access token should be accepted for model deployment.
@seanbetts , did you ever figure this one out? Just encountered this error myself.
I figured this out in my case: Some models require you to accept some T&C or take other action like requesting access, followed by “allowing” that model. In that case, Vertex is not clear/accurate about the error. It still just says “that’s not a valid token” or something - but actually it’s that for the model you’ve chosen, the token doesn’t have access to it yet. What worked for me is to click the link at the top of the Vertex form that takes you to that model on hugging face, followed by taking the required action and allowing my account to have access to it. Then the error went away and I was able to deploy into Vertex. Good luck!
I followed all the steps exactly, created multiple tokens, and tried everything I could, but nothing seems to work.
I’m not sure if we’re supposed to link the GCP account somewhere on the Hugging Face platform—or the other way around—but either way, it’s not working for me.
You have to visit the actual model page on HuggingFace and accept the T&C. Depending on the model, this might require submitting a request for access. This has nothing to do with Google Cloud, per se, it’s a requirement for some models hosted on HuggingFace.
I am running into same problem where it says that token is invalid. I regenerated the token and also the model has no requirement for accepting any terms on hugging face.
I am running into the same problem.
Hmm… Perhaps an issue…?
