The first bit of code runs, but I get the eos_token_id error. The second bit fails. So adding the token to the input_ids doesn’t work. At least the model persists for the second query, so I’m making progress.
Anyway, big thanks to Huggingface for posting these great models. You guys rock!
from transformers import GPTNeoForCausalLM, GPT2Tokenizer
from macos_speech import Synthesizer
model = GPTNeoForCausalLM.from_pretrained("EleutherAI/gpt-neo-1.3B")
tokenizer = GPT2Tokenizer.from_pretrained("EleutherAI/gpt-neo-1.3B")
prompt = (
"In a shocking finding, "
)
input_ids = tokenizer(prompt, return_tensors="pt").input_ids
gen_tokens = model.generate(
input_ids,
do_sample=True,
temperature=0.9,
max_length=100,
)
gen_text = tokenizer.batch_decode(gen_tokens)[0]
print(gen_text)
prompt = (
"Albert Einstein was "
)
input_ids_pre = tokenizer(prompt, return_tensors="pt").input_ids
nput_ids = input_ids_pre + tokenizer.eos_token
gen_tokens = model.generate(
input_ids,
do_sample=True,
temperature=0.9,
max_length=100,
)
gen_text = tokenizer.batch_decode(gen_tokens)[0]
print(gen_text)