T5 randomness in generation

[GenV]

Hello,
I would like to use the T5 network to obtain different outputs with same input, like an intelligent randomness in generation.

Example:

Run1:
Input: input1
Output: output1

Run2:
Input: input1
Output: output2

...

How can I do this?

Thank you

[marshmellow77]

Hi Gennaro

You can use the do_sample parameter to introduce randomness in text generation.

Example:

Once you use sampling, you can even generate multiple responses with one inference request using the num_return_sequences parameter:

Hope that helps!

Cheers
Heiko

[tilomichel]

Hey, i also found this article from huggingface really useful for your problem How to generate text: using different decoding methods for language generation with Transformers

Best regards

[GenV]

@marshmellow77 Thank you, it works fine!
@tilomichel thank you too!

[ppujari]

Where is the usecase of this?