Can t5 be used to text-generation?

Beginners
1

Hello to all,

I’m following this tutorial: https://huggingface.co/blog/how-to-generate

which says: " Auto-regressive language generation is now available for GPT2 , XLNet , OpenAi-GPT , CTRL , TransfoXL , XLM , Bart , T5 in both PyTorch and Tensorflow >= 2.0!"

so I wanted to try to do the same, they just change the model to T5. However even though the model runs, the output is very strange. This is the code:

!pip install transformers
import tensorflow as tf
from transformers import TFT5ForConditionalGeneration, T5Tokenizer

tokenizer = T5Tokenizer.from_pretrained("t5-small")
model = TFT5ForConditionalGeneration.from_pretrained("t5-small", pad_token_id=tokenizer.eos_token_id)
input_ids = tokenizer.encode('Hello, my dog is cute', return_tensors='tf')
greedy_output = model.generate(input_ids, max_length=50)
print("Output:\n" + 100 * '-')

print(tokenizer.decode(greedy_output[0], skip_special_tokens=True))

I get this output:

Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello

Because of this, and taking into account that I have not found many text-generation examples with t5, I would like to ask if this is possible? if so, why my output is so strange?

2 Likes
2

My mistake, reading the documentation is required that the first token is the task, for example ‘summarize’.

3

Right, I know about the task preamble, but I was wondering as well what task to specify for plain old auto-regressive text generation?

2 Likes
4

Yes, using summarize task works. Here is an example I used:

from transformers import T5ForConditionalGeneration, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("t5-base")
model = T5ForConditionalGeneration.from_pretrained("t5-base")
input_ids = tokenizer.encode('summarize: I enjoy walking with my cute dog', return_tensors='pt')
greedy_output = model.generate(input_ids, num_beams=7, no_repeat_ngram_size=2, min_length=50, max_length=100)
print("Output:\n" + 100 * '-')

print(tokenizer.decode(greedy_output[0], skip_special_tokens=True))

Output:

I enjoy walking with my cute dog - he is a joy to walk with and is very affectionate with me. I like to spend time with him on walks with his kitty cat, leo, who is so cute!

5

I want to fine-tune the model for my task. but when I trying to use “transformers/examples/pytorch/language-modeling/run_clm.py” gives this error.

ValueError: Unrecognized configuration class <class 'transformers.models.t5.configuration_t5.T5Config'> for this kind of AutoModel: AutoModelForCausalLM.
Model type should be one of RoFormerConfig, BigBirdPegasusConfig, GPTNeoConfig, BigBirdConfig, CamembertConfig, XLMRobertaConfig, RobertaConfig, BertConfig, OpenAIGPTConfig, GPT2Config, TransfoXLConfig, XLNetConfig, XLMConfig, CTRLConfig, ReformerConfig, BertGenerationConfig, XLMProphetNetConfig, ProphetNetConfig, BartConfig, MBartConfig, PegasusConfig, MarianConfig, BlenderbotConfig, BlenderbotSmallConfig, MegatronBertConfig.

Is there any solutions for this?

6

As the error indicates, you cannot use the run_clm script with a T5 model as there is no version of T5 with a causal LM head. The error listed all the possible architectures you can use instead.

1 Like
7

Hi @sgugger ,

I want to add some small rules when generating the output text.
For e.g:- First number should be larger than the second generating number in the generating sentence.

Is that task is feasible inT5?

8

hello! @sgugger , I’m reproducing a model for a paper and need to use t5-base model for Casual Language Modeling. I find nothing about it, would it be possilble to ask you?