How can I use diverse beam-search? (it isn't working in my code)

🤗Transformers
1

Hello, I want to get several promising generative candidates using the diverse beam-search decoding.
But when I execute the code as follows, they all produce the same sentence.

sample_outputs = self.model.generate(
      input_ids = input_ids,
      max_length = args.max_input_length,
      num_beams = 6,
      num_beam_groups = 3,
      num_return_sequences = 3,
      pad_token_id = self.tokenizer.pad_token_id,
      eos_token_id = self.tokenizer.eos_token_id,
)

If num_return_sequences is set to be the same as num_beam_groups, won’t each group’s sentence be printed one by one?

However, all num_return_sequences sentences returned are the same.

I checked that deleting num_beam_groups returns different sentence candidates well.
I would appreciate it if you could tell me which variable I should add to use the diverse beam-search decoding.

Thanks.

2

Not sure if you’ve already figured this out, since this is somewhat of an old post, but I recently ran into this problem and fixed it by adding the diversity_penalty argument in .generate().

This is what controls the discouragement of similar outputs in each group in the logits processor that is used within the generate code.

See this documentation.

2 Likes