Hi all,
Recently I’m going to do a translation task using the vanilla Transformer model in the paper “Attention is all you need”, but when I search for model list in Transformers docs, there seems no origin transformer. Anyone know where I can find the model for translation task, or I have to implement it manually?
I know that pytorch library has a Transformer class, but it seems that class hasn’t embedding module and isn’t specified for particular task (like BertForxxx etc.).
1 Like
You may prefer to use libraries dedicated for that instead. For instance OpenNMT, fairseq, MarianMT. The latter has also been implemented in transformers.
@BramVanroy I wonder is it possible to use a vanilla Transformer encoder with no any pre-training?
What do you mean? If you do not train a model, it’s weights are randomly initialized. That means that without training, the model will give you random output. You can make use of models that someone else pretrained, though. Models - Hugging Face