I need the probability distribution of word generation to calculate the loss in my original loss function. In particular, the model differs from a normal loss function in that it generates sentences in the same way as when testing. Therefore, I have tried to calculate the loss using model.generate…

I can’t answer many of your questions, but I did find this code snippet useful to get a computational graph with generate(): from undecorated import undecorated from types import MethodType generate_with_grad = undecorated(model.generate) model.generate_with_grad = MethodType(generate_with_grad, m…

How to output loss from model.generate()?

🤗Transformers

Hottuck January 26, 2024, 4:22am 9

I have a question.
Are there concerns that temporarily removing @torch.no_grad() could lead to learning during the test phase?

Topic		Replies	Views
Difference between model.generate() and model() outputs Intermediate	2	3009	March 3, 2024
Generation Probabilities: How to compute probabilities of output scores for GPT2 🤗Transformers	24	28881	April 5, 2023
A potential in-place operation that caused an RuntimeError 🤗Transformers	1	2317	January 19, 2021
[Announcement] Generation: Get probabilities for generated output 🤗Transformers	63	41029	January 20, 2025
Carrying Gradients Through Generate Research	5	2736	January 29, 2023

How to output loss from model.generate()?

Related topics