Help to choose a model for compact summarization

🤗Hub
1

Which language model should be used to summarize a small number of small comments into a single sentence? For example, if this is the input:

“week 1: agent 375 has a good performance
week 2: agent 375 did 3 mistakes in Monday
week 3: agent 375 has the best performance in the team”

I want to have output like this:

“agent 375 performed very well, except few mistakes”

I also want this model to be small enough to run locally with 8GB of RAM.

1 Like
2

There are many models that can be used for text summarization tasks, but with 8GB of RAM, it’s tough to use a general-purpose model larger than 3B. I think it would be better to find a small model that has been adjusted specifically for summarization, or to try a general-purpose model of 2B or less.
There is no way to search by size, so there are also large models mixed in. The T5 models are large.