I’m trying to understand what the summarization pipeline is doing exactly.
I’ve noticed the following:
- When running a model in a simple text generation (using
model.generate()) the output is cut short. - But when running it in summarization pipeline it isn’t cut.
But if I understand correctly, the pipeline cannot get over the model_max_length limit, as it’s not doing recursive summarization / map-reduce / sliding window approach. I’d need to implement those myself, right?
So what is the difference then between txt gen and summarization pipeline?