Use transformer without position embeddings being added?

abpalmarini · June 13, 2021, 2:45pm

Hi,

I can’t seem to find any option to disable position embeddings being added in models like BERT and GPT2. I’d like to use such a model for a non NLP task where position is irrelevant. Is it possible to do this in Hugging Face?

Thanks

Topic		Replies	Views
Learning sets and disabling positional embedding knowledge? Intermediate	0	287	May 10, 2023
`BertEmbeddings` contains positional embedding? 🤗Transformers	2	3005	December 27, 2022
What can transformers learn without position encoding? Research	1	3010	June 10, 2021
Creating a tokenizer with both custom tokens and positions Beginners	5	1174	April 22, 2022
How to use custom positional embedding while fine tuning Bert Beginners	2	2703	September 14, 2022

Use transformer without position embeddings being added?

Related topics