Can I get clarification on what exactly transformers does vs what the model does?

svenpvoigt · April 26, 2025, 2:21am

Hi there,

I am trying to figure out where documentation for models exists. For example, I am looking at the pipeline documentation which says that task is some id. But it is not a user defined id because passing “foo” as the task to the model gemma-3-27b-it gives me an error that lists all the tasks. Is there a function to call that lists the tasks ahead of time without having to get an error message? It is not clear from the documentation that the tasks are implemented by each model not the pipeline api - and it would be nice to know exactly what a model does for implementation purposes rather than some generic description of tasks in the tutorial. I would rather have some way of figuring out what a particular model does so I can implement it. Are there any tools that help me figure this out? Maybe it’s possible to parse it from the config files or the model file?

Also, how can I get information on message formatting for each task? Is there a way to figure this out or is it dependent on the information provided on the model card? So if the tasks and message formats are not listed on the model card, is there a way to determine these? Especially because I am also not seeing any source code implementing a model class that lists tasks and message parsers. Maybe there is a way to parse these from the config or model files as well?

Thanks

John6666 · April 26, 2025, 8:44am

It seems that tasks are being retrieved from classes registered in AutoModel, so you should be able to identify the problem by checking whether the class corresponding to the task is defined in the code.

I’m not sure if there is a simple method (a dedicated function) for this…

github.com/huggingface/transformers

src/transformers/pipelines/init.py

main


      
          
          if task is None and model is not None:
              if not isinstance(model, str):
                  raise RuntimeError(
                      "Inferring the task automatically requires to check the hub with a model_id defined as a `str`. "
                      f"{model} is not a valid model_id."
                  )
              task = get_task(model, token)
          
          # Retrieve the task
          if task in custom_tasks:
              normalized_task = task
              targeted_task, task_options = clean_custom_task(custom_tasks[task])
              if pipeline_class is None:
                  if not trust_remote_code:
                      raise ValueError(
                          "Loading this pipeline requires you to execute the code in the pipeline file in that"
                          " repo on your local machine. Make sure you have read the code there to avoid malicious use, then"
                          " set the option `trust_remote_code=True` to remove this error."
                      )
                  class_ref = targeted_task["impl"]

github.com/huggingface/transformers

src/transformers/models/auto/modeling_auto.py

main

# coding=utf-8
# Copyright 2018 The HuggingFace Inc. team.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Auto Model class."""

import warnings
from collections import OrderedDict

from ...utils import logging

This file has been truncated. show original

svenpvoigt · April 27, 2025, 6:32pm

@John6666 Thanks that’s a good place to start looking!

Also, to add an example to the original post, the jinaai-embeddings model implements all custom tasks but lists them on the model card (e.g., retrieval.query, text-matching). However, it is unclear what the input format should be for each task just from the model card. It looks like lists of strings, but would need to see the model implementation to be sure there aren’t other options.

svenpvoigt · May 6, 2025, 10:42pm

I think I have an answer:

the message format is always a list of strings for the tokenizer, unless the tokenizer includes a template. In that case the template can be printed out with tokenizer.chat_template and usually includes system and user roles as well as some keywords like add_generation_prompt.

However, it doesn’t seem to be overall standardized and there is a lot of custom code for models.

So final answer: most everything has to be explained in the model card and you have to kind of figure out how to make it work from a couple examples.

system · May 7, 2025, 10:43am

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.

Topic		Replies	Views
What docs have I missed? Beginners	0	216	April 26, 2023
Fundamental newbie questions Beginners	1	1329	December 6, 2020
When trying to run model I get model_type is not defined Beginners	3	16	April 13, 2025
Bloom-560m pipeline task parameter invalid Models	3	1078	September 23, 2022
AutoModelForCausalLM and transformers.pipeline Beginners	2	560	August 29, 2024

Can I get clarification on what exactly transformers does vs what the model does?

Related topics