Difference between pipeline and model.generate?

🤗Transformers
1

I tried the following two things and find a significant difference between pipeline and model.generate to complete sequences.

model_pr = AutoModelForCausalLM.from_pretrained("gpt2")
tokenizer.decode(model_pr.generate(**input_tok)[0])
'My name is Merve and my favorite+ and my CR+ and my CR+ and my CR'

(2) Using pipeline to do that same thing

generator = pipeline('text-generation', model='gpt2') 
generator(input) 
[{'generated_text': 'My name is Merve and my favorite brand is Baskin-Robbins.\n\n"They bring a whole lot of stuff to the table and we have to come up with a new way of making a big deal," explains Jeff.'}]

I get a lot more sensible output for pipeline for some reason. My understanding was that both should have given similar responses.

1 Like