When I try to use my fine-tuned Causal LM model to inference a prompt, I get nothing but the last word repeated multiple times

🤗Transformers
1

As the title suggest, every time I fine-tune my model, I get no actual result, I just get the last word repeated multiple times. The model I’m fine-tuning is RoFormer, and here is an example of what is happening:

>> prompt = "Somatic hypermutation allows the immune system to" 
>> generator(prompt)
[{'generated_text': 'Somatic hypermutation allows the immune system to to to to'}]

Here is a bit of details:-

  • I tried multiple datasets.
  • I tried following the official documentation for training a Casual LM.
  • The training, and validation losses are also really small.
  • The base language is Chinese.

What am I missing?

2

It might be the way the model is fine-tuned (how dataset is structured, how data is processed by data collator and trainer, etc.) but repetition like the example you’ve provided also seems to be a fairly common occurrence. There exists a repetition_penalty parameter that you can try toying with: Transformers - repetition_penalty parameter

Out of curiosity, is there a particular reason you’re trying to generate English text with a model whose base language is Chinese?