PretrainedConfig example to use it in GPT2 text-generation pipeline

🤗Transformers
1

I want to modify the default parameters in the PretrainedConfig and use it in the pipeline,

prediction = pipeline('text-generation', model=model_path, tokenizer=tokenizerObject)

Can someone please help me in constructing the PretrainedConfig object with parameters so that I can use it in the pipeline object.

2

Hi @bala1802, since the TextGenerationPipeline accepts a pretrained model as an argument perhaps you can adapt the following for your use case:

from transformers import AutoModelWithLMHead, AutoTokenizer, AutoConfig, TextGenerationPipeline

model_ckpt= "gpt2"
# override default parameters here, eg output the hidden states
config = AutoConfig.from_pretrained(model_ckpt, output_hidden_states=True)
model = AutoModelWithLMHead.from_pretrained(model_ckpt, config=config)
tokenizer = AutoTokenizer.from_pretrained(model_ckpt)

pipe = TextGenerationPipeline(model=model, tokenizer=tokenizer)

You might not even need the TextGenerationPipeline class since I think that this is equivalent to instantiating the pipeline as follows:

pipe = pipeline('text-generation', model=model, tokenizer=tokenizer)