I just deployed dolphin2.2-mistral using hugging face DEPLOY->INFERENCE ENDPOINT and got it working. It does take inputs as payload and working fine. My issue is, what value should I send to have GPT like experience? Something like system, user and assistant. How can I change the system prompt, an…

Sending System and User Prompt in Deployed endpoint

upgiant November 17, 2023, 4:39pm 3

Do you have example of payload? I am pretty new and I am sure I am going to spend a lot of time on this which I am not sure I would be able to.

Topic		Replies	Views
Handler.py not executed in Inference Endpoint Inference Endpoints on the Hub	0	265	September 13, 2023
Help with custom handler.py for model inference endpoint Beginners	1	745	February 24, 2024
Using Langchain ChatHuggingface with Text Generation Inference: missing field `inputs` Beginners	2	737	August 8, 2024
Guide/Tutorial to write an inference endpoint for custom models Inference Endpoints on the Hub	5	1806	October 19, 2024
How do I send system prompts using inference api serverless, llama3 8b instruct model Beginners	2	1323	April 29, 2024