Looking for a Translation Model for English to 100+ Languages, Comparable to DeepL or Google, for Local Deployment

Research
1

Hello everyone,

I am working on a project where I need to translate text from English into over 100 different languages. The translation quality needs to be comparable to services like DeepL or Google Translate.

Is there a model available that meets these requirements and can be run locally without the need for external APIs? Additionally, does this model support translating HTML source code and WordPress posts?

Python compatibility would be ideal as it’s my primary working environment.

Thanks in advance for any help and guidance.

Best regards,
BaGRoS

2

Facebook research released a paper called “No Language Left Behind,” which open sources some machine translation models. Most of the models range from 600M to 3.3B parameters, which you might be able to be run locally. I doubt they can translate HTML source code and WordPress posts, but they should do well for natural languages.
Paper link:

Models repo:

2 Likes
3

Hi,
2 months ago (august 2023) Facebook has released seamless;
here is the model repo in HF: facebook/seamless-m4t-large · Hugging Face
But I don’t think that you can run it locally

1 Like
4

That does appear to be locally hostable, but it is not exactly straight-forward for new users. I think it would be worthwhile to search around online for guides on how to use seamless-m4t. That does seem like the best project for what the OP asked for.

5

Well i checked some cases for all models except 54B. It works bad for one word sentences.
So when i try translate “кардиган” which means “cardigan” in russian, facebook/nllb-200-3.3B translates as “I’m wearing a cardigan.”.