Change Positional Embedding in T5 from Relative to Absolute

mab · May 25, 2022, 11:26am

I would like to change the positional embedding in T5 from relative to absolute, anyone can direct me on an easy way of doing it or I need to dive in to the source code and change it?

Topic	Replies	Views
Postional Encoding calculation for T5 🤗Transformers	180	March 16, 2023
Can we use t5 for decoder only? Beginners	186	August 17, 2023
Use transformer without position embeddings being added? Beginners	841	June 13, 2021
Changing the embedding layer in a quantized model Beginners	59	June 30, 2024
Turn word embedding to word id (using T5 decoder) 🤗Transformers	334	January 8, 2022

Change Positional Embedding in T5 from Relative to Absolute

Related topics