Whole day I have worked with available text generation models
Here you can find list of them : Models - Hugging Face
I want to generate longer text outputs, however, with multiple different models, all I get is repetition.
What am I missing or doing incorrectly?
I will list several of them
Freshly released meta GALACTICA - facebook/galactica-1.3b · Hugging Face
The code example
from transformers import AutoTokenizer, OPTForCausalLM
tokenizer = AutoTokenizer.from_pretrained("facebook/galactica-1.3b")
model = OPTForCausalLM.from_pretrained("facebook/galactica-1.3b", device_map="auto")
input_text = "The benefits of deadlifting\n\n"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")
outputs = model.generate(input_ids,new_doc=False,top_p=0.7, max_length=1000)
print(tokenizer.decode(outputs[0]))
The generated output
Facebook opt - facebook/opt-350m · Hugging Face
The tested code
from transformers import GPT2Tokenizer, OPTForCausalLM
model = OPTForCausalLM.from_pretrained("facebook/opt-350m")
tokenizer = GPT2Tokenizer.from_pretrained("facebook/opt-350m")
prompt = "The benefits of deadlifting can be listed as below:"
inputs = tokenizer(prompt, return_tensors="pt")
# Generate
generate_ids = model.generate(inputs.input_ids, max_length=800)
tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]
The generated output
