Help Improving Abstractive Summarization

🤗Transformers
1

Hey everyone,

Hope you’re doing great!
I’m a beginner when it comes to using transformers.
I’ve been working on book summarization project for a while, the idea is to split the book into chapters then the chapter into chunks and summarize the chunks separately.
I’ve tried several models and the summaries provided aren’t that good. Some of the problems are:

  • Some sentences aren’t fully generated.
  • The context is lost most of the time.

Do you guys have any suggestions please?
Also, I have found these two papers:

  1. Mind The Facts: Knowledge-Boosted Coherent Abstractive Text Summarization
  2. Enhancing Factual Consistency of Abstractive Summarization

But I couldn’t find any code implementation of these models. If you have any implementations please let me know.

Thank you for your time!
Have a good day.

2

hey @haithembrr, your approach sounds very sensible :slight_smile:

have you tried playing around with the parameters of the model’s generate function, e.g. max_len and trying different strategies like beam search vs sampling (see docs)?

alternatively, you could have a look at the discussion in this thread to see if someone has also run into the same problem: Summarization on long documents

1 Like
3

Thank you so much for your feedback @lewtun ! I highly appreciate it.
Actually I didn’t experiment with generation function parameters! I’ll make sure to do so and will get back to you as soon as have some results! :grinning_face_with_smiling_eyes:

1 Like