How to improve the MNMT inference time?
If you are using cloud computing, you can use EIA as a cheap solution. Need more information on how you are doing inference now
Hi,
Thanks for ur response.
I m using ctranslate2 inference engine.
How does it relate to transformers then?
I trained Multilingual neural machine translation using transformer architecture with fairseq toolkit. I converted fairseq model to ctranslate2 model using ctranslate2 converter. I want to improve the inference time of my model. Can you pls help?
But you are not using any of the Hugging Face tools? These forums are not for the architecture transformer but for the libraries of Hugging Face, amongst which the transformers package.
You’ll have better look asking this question on the Fairseq and OpenNMT forums.