Is there any way to detect the language of the given input text?
There are many models to translate form one language to other.
It should be fairly easy to train a small model to detect languages, not sure if there is one already but seems a bit overkill.
I have used this library in the past to detect language: https://pypi.org/project/langdetect, it should be more than enough as a pre-processing step to then choose which translation model to use (if that is the use case you were referring to)
Yeah i saw the package which you mentioned but I am looking for something with the transformer based model.
Well I am not aware of any pretrained model that does what you ask (there is one to identify programming languages ). You could train your own model to do so, something like multilingual distilbert, but again, seems a bit overkill.
There are already a number of ways to get to solve this problem. We can use standard libraries. I have got a solution if you still want help in using non-transformer solutions you may get in touch with me.
I am new I do not know if we can take names of other libraries in this forum.
The transformer library is better used for other stuff but if we get a solution to this issue inside the transformer itself then we do not need other libraries, which is obviously of advantageous.
Hope this helps.
You can use textblob and fasttext libraries. Saw someone mentioning other libraries so just posting.