Transformer for very big text

So I have some ~1200 data consisting of assembly language which has on average of ~15K words per line which is indeed very big data and I want to classify them into two classes (they are labelled)

Any method I can achieve my goal? as the transformers max length limit is mostly 256 or 512.

Thanks.

1 Like

Longformer might help you