Using .generate with TAPAS as encoder in EncoderDecoder

I’m trying to to train the model to perform text generation conditioned on tables.

Since TAPAS can encode the semi-structured meaning in tables, I guessed it was a good choice to use it as an encoder and say GPT2 (or any other CLM) as a decoder.

I however encountered a problem when trying to generate from that EncoderDecoder model, this:

I guess this is since model.generate() for EncoderDecoder does not expect to have the extra dimension of token_type_ids that TAPAS has.
Can anyone think of a way I can make this work?


Hi, does anyone know?


I’ll investigate and get back to you.

Update: it works for me when overriding the _update_model_kwargs_for_generation method. The token_type_ids shouldn’t be updated, as a table only needs to get encoded once.

Notebook: Google Colab

It works for me!
I’ll now go ahead and try to train it for conditional generation.
Thanks! :pray: