My understanding is that when passed a sequence of input vectors, a transformer self-attention block computes three different transformed versions of that sequence: the keys, the queries, and the values. Then it takes the key/query dot products, softmaxes, and takes a weighted average of the values. …

Past_key_values - why not past_key_values_queries?

cwaki7 May 28, 2023, 6:12am 5

(post deleted by author)

Topic		Replies	Views
Role of past_key_value in self attention Intermediate	0	721	November 23, 2021
Outputs change if re-using KVCache (past_key_values) for model.forward and generation 🤗Transformers	5	285	January 22, 2025
Isn't KV cache influenced by position encoding in inference? 🤗Transformers	3	927	May 16, 2024
Correct input_ids when passing past_key_values 🤗Transformers	2	939	June 14, 2024
KV cache sizing 🤗Transformers	0	769	August 24, 2023