I know I can intercept the first 512 lengths.But I don’t want to do this. This task can understand the classification of texts. AutoModelForSequenceClassification can be directly used for classification, but the question is whether the input can be some 512 length sentences, and use pooler layer t…

My input sentence is very long(more than 512). What should I do when I want to fintune model about classify?Thanks

ccfeidao September 3, 2021, 4:49am 4

I want to user BigBird to try my task, but have a new problem, I am very Confusion.

Topic		Replies	Views
Token Classification Models on (Very) Long Text Models	8	7194	March 9, 2023
How do I search for sentence transformer models by context window/token length/word piece count? 🤗Hub	0	105	March 27, 2024
Model for Long Text Classification Beginners	0	126	February 20, 2024
Text classification training on long text Intermediate	1	1957	January 29, 2024
How to train with very long sequences? Beginners	2	471	May 20, 2022