ERROR: openai.UnprocessableEntityError: Error code: 422

Beginners
1

I ran my custom fine-tuned Mistral 7b Instruct on hf inference and want to use openAI client, so that the API won’t return whole text. Below is settings of api (With TensorFlow it fails to run)

CleanShot 2024-05-09 at 19.14.53@2x
CleanShot 2024-05-09 at 19.14.53@2x1662×1736 362 KB

But now code gives me this error:

Traceback (most recent call last):
  File "/Users/almatkairatov/Desktop/GENERAL/CreateML training/ML/huggingFace/main.py", line 9, in <module>
    chat_completion = client.chat.completions.create(
  File "/opt/miniconda3/envs/huggingFace/lib/python3.9/site-packages/openai/_utils/_utils.py", line 277, in wrapper
    return func(*args, **kwargs)
  File "/opt/miniconda3/envs/huggingFace/lib/python3.9/site-packages/openai/resources/chat/completions.py", line 590, in create
    return self._post(
  File "/opt/miniconda3/envs/huggingFace/lib/python3.9/site-packages/openai/_base_client.py", line 1240, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
  File "/opt/miniconda3/envs/huggingFace/lib/python3.9/site-packages/openai/_base_client.py", line 921, in request
    return self._request(
  File "/opt/miniconda3/envs/huggingFace/lib/python3.9/site-packages/openai/_base_client.py", line 1020, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.UnprocessableEntityError: Error code: 422 - {'error': 'Template error: template not found', 'error_type': 'template_error'}

This code I just take for testing, but as I said it is not working

import os
from openai import OpenAI

# init the client but point it to TGI
client = OpenAI(
    base_url=os.path.join("https://url.endpoints.huggingface.cloud", "v1/"),
    api_key="API_KEY",
)
chat_completion = client.chat.completions.create(
    model="tgi",
    messages=[
        {"role": "user", "content": "Why is open-source software important?"}
    ],
    stream=True
)

# iterate and print stream
for message in chat_completion:
    print(message.choices[0].delta.content, end="")
2

Hi,

It looks like no tokenizer_config.json with a chat template is present. Does your repository contain that? Similar to this line: https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2/blob/main/tokenizer_config.json#L42

1 Like
3

yeah, I have found that right before your answer, thank you for your response.

4

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.