I need the probability distribution of word generation to calculate the loss in my original loss function. In particular, the model differs from a normal loss function in that it generates sentences in the same way as when testing. Therefore, I have tried to calculate the loss using model.generate…

I can’t answer many of your questions, but I did find this code snippet useful to get a computational graph with generate(): from undecorated import undecorated from types import MethodType generate_with_grad = undecorated(model.generate) model.generate_with_grad = MethodType(generate_with_grad, m…

How to output loss from model.generate()?

🤗Transformers

mittu April 22, 2022, 5:37am 3

Thanks for sharing your simple solution!

I’ll give this method a try!

Topic		Replies	Views
Difference between model.generate() and model() outputs Intermediate	2	3007	March 3, 2024
Generation Probabilities: How to compute probabilities of output scores for GPT2 🤗Transformers	24	28880	April 5, 2023
A potential in-place operation that caused an RuntimeError 🤗Transformers	1	2316	January 19, 2021
[Announcement] Generation: Get probabilities for generated output 🤗Transformers	63	41024	January 20, 2025
Carrying Gradients Through Generate Research	5	2736	January 29, 2023

How to output loss from model.generate()?

Related topics