EncoderDecoderModel with Longformer and Bert

🤗Transformers
1

Hi,
Is it possible to create EncoderDecoderModel using Longformer and Bert? I tried Longformer and Roberta and it works (the training runs), but if I use Longformer and Bert, I get this error message RuntimeError: CUDA error: device-side assert triggered when I train it.

2

Normally Longformer and BERT should work in an encoder-decoder setting.

If you have a CUDA error, it’s advised to test the code on CPU and see if you’re getting an error that is more interpretable.

1 Like