Translating multiple languages to English (Tensorflow) - repost

🤗Transformers
1

Hello there!

I am trying to use tensorflow to translate from many different languages to English but I am not able to adapt the little examples available in the docs (which are written in pytorch)

See below:

import sentencepiece
import transformers
from transformers import T5TokenizerFast, TFT5ForConditionalGeneration

article_fr = "bonjour je voudrais un camembert."

model = TFT5ForConditionalGeneration.from_pretrained("t5-small", return_dict = True)
tokenizer = T5TokenizerFast.from_pretrained("t5-small")

My two attempts at translating fail miserably

encoded_hi = tokenizer("translate French to German: "+article_fr, return_tensors="tf")
generated_tokens = model.generate(**encoded_hi)
tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)

basically returns my input in French

['bonjour je voudrais un camembert.']

Same thing with a pipeline

[{'translation_text': 'Bonjour je voudrais un camembert.'}]

Do you know what the issue is? I am open to use any other model, my only requirements are to be able to translate both japanese and french to english in tensorflow

Any help greatly appreciated!
Thanks!!!

2

I would suggest you to use a model trained on that task instead, like OPUS models.

!pip install transformers
!pip install sentencepiece
from transformers import pipeline
translation = pipeline("translation", "Helsinki-NLP/opus-mt-fr-de")
tr_text = translation("bonjour je voudrais un camembert.")
tr_text[0]["translation_text"]
## Hallo, ich hätte gern einen Camembert.
1 Like