Hello! I was wondering what changes I would need to make to use GRPOTrainer with a custom PyTorch module class. Currently I have an nn.Module subclass that wraps around an existing Huggingface transformer, except with a custom forward and generate function. I was wondering though if there were res…

Using GRPOTrainer with a custom PyTorch module?

enyoukai April 29, 2025, 4:44pm 4

fixed. turns out i just need to pad back the output with the original prompt lol

1 Like