Is summary of 1024 tokens not useless?

I am trying to use summarization with the pipeline transformers but I can’t use an input > 1024 tokens, whatever the model, which is barely the lenght of a summary.

I guess summarization is to extract information from a long text, not from a couple of lines that we can read in 2 minutes. Maybe I am missing something ? Because a summary of a text of 1024 tokens seems for me quite useless…

Look into the models meant for long sequences (bigbird, longformer, long t5)
google/bigbird-pegasus-large-bigpatent
allenai/led-large-16384-arxiv
pszemraj/long-t5-tglobal-base-16384-book-summary

1 Like