Assume that I’m using the PEGASUS model for conditional generation.
Is it possible to run the encoder part and decoder part separately without manually writing the generation script?
For encoder part, I know I can do something like this to get the last hidden state:
model = PegasusForConditionalGenereation.from_pretrained(...) encoder_output = model.model.encoder(**input)["last_hidden_state"]
Then I will use
encoder_output for something else, and optionally run the decoder of the model for sequence generation. The rest of the
model contains the
lm_head for token generation, but I haven’t found a way to easily run the generation. I don’t want to just do another
model.generate(...) because that will run the encoder part again, which is a waste of time.
Is it something achievable easily with some methods in transformers library? Thanks!