How to read/use LLM model templates

Beginners
1

Hi guys
Actually as subject. What are the templates of models. And how actually to read them and to use?

  • is it some kind of real go templates or something or is it just an example to grasp the structure idea?
  • Are these for parsing/formatting purposes?
  • What about the context data (e.g. .Role, .Content) is it more like to be injected while formatting or placeholders to be scanned?
  • Are these quite standardized in language/syntax or fluent for any other model

Thank you

1 Like
2

The apply_chat_template function is probably more primitive than you think. It literally just takes a text template and puts it together with the model, and at runtime the tokenizer reads it and converts the OpenAI-style format list[dict] to str.

I don’t think the OpenAI-style format is standardized in any way, it’s just something that’s convenient to use. To be precise, it’s a little different for each software…
Well, it’s easy to use.

3

Ah great. Thanks.
So it’s just a matter of serialization of structured messages into a text input

Still the question around the standards. It was mentioned in referred docs it uses Jinja as a templater. Yet it seems not all the models follow this as well.
E.g. Gemmas something go-like style

{{- range $i, $_ := .Messages }}

so how apply_chat_template knows how to render them in general?

And regarding input structure it self. Are at least .messages .role and .content are some how basic structure or these could vary as well?

1 Like
4

Even though it is an OpenAI-style format, it is not an OpenAI class. It must be a list[dict] with a similar feel…

For more information on the template format, see below.

chat = [
  {"role": "user", "content": "Hello, how are you?"},
  {"role": "assistant", "content": "I'm doing great. How can I help you today?"},
  {"role": "user", "content": "I'd like to show off how chat templating works!"},
]