How to implement bind_tools to custom LLM from huggingface pipeline(Llama-3) for a custom agent

Intermediate
1

Code

name = "meta-llama/Meta-Llama-3-8B-Instruct"

auth_token = ""

tokenizer = AutoTokenizer.from_pretrained(name,use_auth_token=auth_token)

bnb_config = BitsAndBytesConfig(
    load_in_8bit=True,
)

model_config = AutoConfig.from_pretrained(
    name,
    use_auth_token=auth_token,
    tempreature=0.1,
)

model = AutoModelForCausalLM.from_pretrained(
    name,
    trust_remote_code=True,
    config=model_config,
    quantization_config=bnb_config,
    device_map='auto',
    use_auth_token=auth_token,
)

streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, max_new_tokens=4096, device_map="auto", streamer = streamer)
llm = HuggingFacePipeline(pipeline=pipe)
chat_model = ChatHuggingFace(llm=llm)

@tool
def some_custom_tool(input_string: str) -> str:
    """Executes some work and returns a success message if successfull else it return the error message"""
    return "SUCCESS"


tools = [some_custom_tool]
prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            f"""
                You are an Assistant......
            """,
        ),
        ("user", "{input}"),
        MessagesPlaceholder(variable_name="agent_scratchpad"),
    ]
)
            
llm_with_tools = chat_model.bind_tools(tools)

            

agent = (
    {
        "input": lambda x: x["input"],
        "agent_scratchpad": lambda x: format_to_openai_tool_messages(
            x["intermediate_steps"]
        ),
    }
    | prompt
    | llm_with_tools
    | JsonOutputParser()
)
            

agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True, return_intermediate_steps= True)

agent_executor.invoke(
    {
        "input": question
    }
)

The issue is that the agent does not execute the tool.
It just returns this but it does not execute the function. Is this a bug in langchain_huggingface because i have tried the same approach with OpenAI and it works.

{
“action”: “some_custom_tool”,
“action_input”: “some input”
}

3 Likes
2

Any luck with the above error ?

1 Like
3

Have you found any solution ?

1 Like
4

Many models seem to require an unofficial chat template for function calling (tool calling).