Hi all,
Recently I’m going to do a translation task using the vanilla Transformer model in the paper “Attention is all you need”, but when I search for model list in Transformers docs, there seems no origin transformer. Anyone know where I can find the model for translation task, or I have to implement it manually?
I know that pytorch library has a Transformer class, but it seems that class hasn’t embedding module and isn’t specified for particular task (like BertForxxx etc.).
You may prefer to use libraries dedicated for that instead. For instance OpenNMT, fairseq, MarianMT. The latter has also been implemented in transformers.
@BramVanroy I wonder is it possible to use a vanilla Transformer encoder with no any pre-training?
What do you mean? If you do not train a model, it’s weights are randomly initialized. That means that without training, the model will give you random output. You can make use of models that someone else pretrained, though. Models - Hugging Face