Hi, for some models there is this tie_word_embeddings parameter. I think it is for the text 2 text models. Can someone please explain what exactly this parameter is doing? Many thanks Philip

What is the `tie_word_embeddings` option exactly doing?

NightMachinery October 15, 2022, 5:17pm 4

Is Embedding*Embedding^{T} = I necessarily true-ish?

Topic	Replies	Views
Should we use tie_weights always? (intend to separate source/target embedding) Beginners	727	November 17, 2021
Using tie_weights() always? (intend to separate source/target embedding) 🤗Transformers	4801	November 17, 2021
Tied weights for encoder and decoder vocab matrix hard coded in T5? 🤗Transformers	824	April 24, 2023
How to understand the bias term in language model head (when we tie the word embeddings)? 🤗Transformers	911	September 12, 2022
How does "_tied_weights_keys" work? Beginners	374	January 3, 2025