Using .generate with TAPAS as encoder in EncoderDecoder

Intermediate
1

Hi,
I’m trying to to train the model to perform text generation conditioned on tables.

Since TAPAS can encode the semi-structured meaning in tables, I guessed it was a good choice to use it as an encoder and say GPT2 (or any other CLM) as a decoder.

I however encountered a problem when trying to generate from that EncoderDecoder model, this:

image
image597×651 195 KB

I guess this is since model.generate() for EncoderDecoder does not expect to have the extra dimension of token_type_ids that TAPAS has.
Can anyone think of a way I can make this work?

Thanks!

2

Hi, does anyone know?
@NielsRogge?

3

Hi,

I’ll investigate and get back to you.

4

Update: it works for me when overriding the _update_model_kwargs_for_generation method. The token_type_ids shouldn’t be updated, as a table only needs to get encoded once.

Notebook: Google Colab

5

It works for me!
I’ll now go ahead and try to train it for conditional generation.
Thanks! :pray: