How to implement bind_tools to custom LLM from huggingface pipeline(Llama-3) for a custom agent

John6666 · June 9, 2025, 2:35am

Many models seem to require an unofficial chat template for function calling (tool calling).

github.com/huggingface/transformers

Is apply_chat_template support function call usage?

opened 06:47AM - 22 Jul 24 UTC

closed 08:07AM - 05 Oct 24 UTC

FanZhang91

bug

### System Info transformers= 4.42.4 python=3.10.13 ubuntu=20.04 ### Who… can help? _No response_ ### Information - [ ] The official example scripts - [ ] My own modified scripts ### Tasks - [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...) - [ ] My own task or dataset (give details below) ### Reproduction I use simple test code to verify the support of the apply_chat_template interface for the function call mode, but result shows input tools information was not added to the output prompt. Is there a problem with my usage？ ```python from transformers import AutoTokenizer get_current_weather = { "type": "function", "function": { "name": "get_current_weather", "description": "Get the current weather in a given location.", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA", }, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}, }, "required": ["location"], }, } } messages = [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "help me query currnet weather in San Francisco."}, ] tokenizer = AutoTokenizer.from_pretrained("xxx/Qwen2-1.5B-Instruct") prompt = tokenizer.apply_chat_template( messages, tools=[get_current_weather,], tokenize=False, add_generation_prompt=True ) print("prompt: ", prompt) ``` prompt: <|im_start|>system You are a helpful assistant.<|im_end|> <|im_start|>user help me query currnet weather in San Francisco.<|im_end|> <|im_start|>assistant ### Expected behavior None

Topic		Replies	Views
AutoModelForCausalLM() to HuggingFaceLLM Beginners	2	2919	October 4, 2024
HF pipelines for simulating user-agent conversations Intermediate	0	52	November 19, 2024
Llama 2 don't reponse prompt invokes Models	0	404	February 9, 2024
AutoModelForCausalLM and transformers.pipeline Beginners	2	595	August 29, 2024
Accelerating inference for local HuggingFacePipeline of Llama3 🤗Transformers	0	87	August 1, 2024

How to implement bind_tools to custom LLM from huggingface pipeline(Llama-3) for a custom agent

Related topics