Not sure if you’ve already figured this out, since this is somewhat of an old post, but I recently ran into this problem and fixed it by adding the diversity_penalty
argument in .generate()
.
This is what controls the discouragement of similar outputs in each group in the logits processor that is used within the generate code.
See this documentation.