How to use Transformer XL for sequence classification?

Can you post the full error message?

As noted here, TransformerXL is the only model in the library that is not supported by the Trainer (you would need to overwrite it).