Are dropout layers activated when calling model.generate()?

[jiaweihuang]

Hi, I’m curious about whether the dropout is activated when we call model.generate()?

For example, if the model is a T5 model loaded by

model = T5ForConditionalGeneration.from_pretrained('google-t5/t5-base')

By default, the dropout layers in T5 model has p=0.1. I wonder if anyone knows whether the dropout layers are activated when we use model.generate() to generate outputs?

Or maybe model.eval() is called somewhere inside model.generate()?

[nielsr]

Hi,

No dropout is deactivated since from_pretrained puts a model in evaluation mode by default (i.e. model.eval() is called on the model).