Hi, I’m currently stucked in getting logits from model.generate. I’m wondering if it is possible to get logits of each seqeucne returned by model.generate. (like logits for each token returned by model.logits)
Hi, one way to do that is to modify generation_utils
For examples, we can extract logits of greedy search in the scores variable (there are many options on next-token search eg. greedy, beam or sample searches)