How I fine-tune BART for summarization using large texts?

[thiagocmoreira]

Good night!

I’m using a pre-trained Bart for summarization and I have my own dataset for fine-tuning (which has a set with the big text and its respective summary). Despite this, my input texts are approximately 2500 characters long and the maximum Bart accepts is 1024. Is there any technique I can use to use all text? I thought of splitting each cell into smaller texts (max 1024) and assigning the same summary to each. Makes sense?

Example:

Before:
ABC: summary1
DEF: summary2

After:
A: summary1
B: summary1
C: summary1
D: summary2
E: summary2
F: summary2

Thanks in advance!

[valhalla]

Hi, there’e already thread for this, you might find it helpful

Summarization on long documents

You can try extractive summarisation followed by abstractive. In the extractive step you choose top k sentences of which you choose top n allowed till model max length. Another way is to use successive abstractive summarisation where you summarise in chunk of model max length and then again use it to summarise till the length you want. This method will be super expensive. You can also combine first + second method.

[thiagocmoreira]

Do you have any idea how I can do this extractive summarization before? I would have to cut my text in half to be the ideal size, but I don’t know how to get the most relevant sentences in this extractive step.

[valhalla]

could you post this question in that thread, people there might have tried this, let’s keep the long summ discussion in one thread