HuggingFace Endpoint Error on AWS

Inference Endpoints on the Hub
1

Hi Everyone,
I have tried to create hugging face endpoint deployed on AWS Nvidia 4GPU, 64 GB. However initiation got failed stating the below error

File "/usr/local/lib/python3.10/dist-packages/transformers/pipelines/base.py", line 1271, in check_task

raise KeyError(

KeyError: "Unknown task custom, available tasks are ['audio-classification', 'automatic-speech-recognition', 'conversational', 'depth-estimation', 'document-question-answering', 'feature-extraction', 'fill-mask', 'image-classification', 'image-feature-extraction', 'image-segmentation', 'image-to-image', 'image-to-text', 'mask-generation', 'ner', 'object-detection', 'question-answering', 'sentiment-analysis', 'summarization', 'table-question-answering', 'text-classification', 'text-generation', 'text-to-audio', 'text-to-speech', 'text2text-generation', 'token-classification', 'translation', 'video-classification', 'visual-question-answering', 'vqa', 'zero-shot-audio-classification', 'zero-shot-classification', 'zero-shot-image-classification', 'zero-shot-object-detection', 'translation_XX_to_YY']"

Application startup failed. Exiting.

Kindly, Help me in this issue.

Thank you.

2

hi @Nithish-2001
I don’t know how you try to deploy it but if you use ui.endpoints.huggingface.co, you might select another task from “Advanced configuration” instead of custom.

4

Hi @mahmutc ,
Thanks for the reply.
I have fine tuned llama-3.1-8B-Instruct model and saved in my hugging face models account.
For endpoint creation I am using this url
https://ui.endpoints.huggingface.co/Nithish-2001/endpoints
Created a new endpoint for my custom fine tuned model → public, task - custom → deployed on aws server

I will modify task to Question-Answering and try it