As of transformers v4.44, default chat template is no longer allowed

John6666 · January 7, 2025, 5:24pm

Perhaps this?
https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html#chat-template

chujiezheng/chat_templates/blob/main/chat_templates/llama-2-chat.jinja

{% if messages[0]['role'] == 'system' %}
    {% set system_message = '<<SYS>>\n' + messages[0]['content'] | trim + '\n<</SYS>>\n\n' %}
    {% set messages = messages[1:] %}
{% else %}
    {% set system_message = '' %}
{% endif %}

{% for message in messages %}
    {% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}
        {{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}
    {% endif %}

    {% if loop.index0 == 0 %}
        {% set content = system_message + message['content'] %}
    {% else %}
        {% set content = message['content'] %}
    {% endif %}

    {% if message['role'] == 'user' %}
        {{ bos_token + '[INST] ' + content | trim + ' [/INST]' }}

This file has been truncated. show original

Topic		Replies	Views
Issue with LlaMA-2 Chat Template (and out of date documentation) Models	7	11859	November 10, 2023
Llama inference with apply_chat_template Beginners	0	204	November 30, 2024
How to stop LLM from going up to the max token limit? Intermediate	1	107	September 25, 2024
When I using the chat_template of llama 2 tokenizer the response of IT model is nothing 🤗Tokenizers	0	110	July 13, 2024
Cannot use chat template functions because tokenizer.chat_template is not set Beginners	1	877	January 27, 2025

As of transformers v4.44, default chat template is no longer allowed

Related topics