Hi there, In Huggingface transformers lib, is there implementation of the original transformers model mentioned in the paper, “Attention is all you need”. I just browsed the documentation site and saw the implementation like BERT, T5, etc., but I didn’t see the original model. Thanks for the clar…

Original transformers model implementation

MickyMike June 1, 2022, 5:11am 3

Yes @Vasily
I used the vanilla transformer implemented in PyTorch.

Topic		Replies	Views
Has vanilla transformer implemented in transformers library? 🤗Transformers	3	1950	June 5, 2022
Reproduce attention is all you need Beginners	0	485	June 25, 2022
How to make pure transformer model Beginners	0	141	May 22, 2024
Vanilla Transformer Beginners	1	1180	June 6, 2023
How pretrained models are trained? Beginners	3	272	October 2, 2020