How to Finetune and deploy LLaVA-1.6

Models
1

Hi,
Im new here and I’m facing the following issues:

  1. I was playing with llava-v1-6-vicuna-13b. Having looked at the github repo with info on how to fine-tune the same, I was able to finetune the model. I uploaded the model to Huggingface on a private repo and I used the deploy using interface endpoints feature but got the following error:

huggingface_hub.utils._errors.RepositoryNotFoundError: 401 Client Error.
Repository Not Found for url:
Please make sure you specified the correct repo_idandrepo_type.
If you are trying to access a private or gated repo, make sure you are authenticated.
Invalid username or password.

  1. When I was using the LlavaNextProcessor library from this link, I was unable to use it on my fine-tuned model and there are no instructions on how to fine-tune using the Library if possible or any other ways.

Any help would be highly appreciated

2

Hi, could you please share the github repo that you used to fine-tune llava 1.6

3

I used the following Repo: