Hi, I am beggining with llama 2 7b chat. I was using it with LangChain, but I get empy value:
import os
os.environ['HUGGINGFACEHUB_API_TOKEN'] = "token here"
from langchain_community.llms.huggingface_pipeline import HuggingFacePipeline
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
model_id = "meta-llama/Llama-2-7b-chat"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, max_new_tokens=500)
hf = HuggingFacePipeline(pipeline=pipe)
from langchain.prompts import PromptTemplate
template = """Question: {question}
Answer: Let's think step by step."""
prompt = PromptTemplate.from_template(template)
chain = prompt | hf
question = "What is AI?"
# ------------ empty ------------------
print(chain.invoke({"question": question}))
# -------------------------------------
I tried it with “TinyLlama/TinyLlama-1.1B-Chat-v1.0” and I got the prompt invoke, but don’t with llama 2.
I used HuggingFaceHub recently and got the message that some models need a pro subscription, does llama 2 need a subscription when I use transformers?
Thanks.