How to make transformer (T5) for translation return n translation inferences?

🤗Transformers
1

So I am trying to use this transformer from huggingface Translation. The issue is that I want n translations returned and not just one. How can I do that? I mean, I want to have ordered translations, that means the translation with index 0 would have the highest confidence, this is important for my use case, which is about translating natural language to commands language (about 40 commands without subcommands).

The github repo and exact model is this one text-to-text-transfer-transformer/t5/models/hf_model.py at main · google-research/text-to-text-transfer-transformer · GitHub

This is the HuggingFace API:

translator = pipeline("translation_xx_to_yy", model="my_awesome_opus_books_model")
translator(text)

But I am intending to use the model directly from the google search github repo, so it seems some tweaking should be done here:

predictions = []
    for batch in dataset:
      predicted_tokens = self._model.generate(
          input_ids=self.to_tensor(batch["inputs"]), **generate_kwargs
      )
      predicted_tokens = predicted_tokens.cpu().numpy().tolist()
      predictions.extend(
          [vocabs["targets"].decode(p) for p in predicted_tokens]
      )

    for inp, pred in zip(inputs, predictions):
      logging.info("%s\n  -> %s", inp, pred)

    if output_file is not None:
      utils.write_lines_to_file(predictions, output_file)

Thanks in advance! Also any suggestion on some other model option to solve this natural language to cmd is welcomed!

2

There’s the num_beams param you could check out that looks for multiple possible translations. This considers multiple top hypotheses.

Then the num_return_sequences in model.generate will generate multiple possible sequences which you could then decode.

3

Yes, but that normalize_logits=True in compute_transition_scores should be the case, because I don’t want the same input length for sure.