Hugging Face Forums
Storing and loading KV cache
🤗Transformers
oran-sh
August 6, 2024, 1:08pm
5
That’s great, any estimate when this PR will be merged?
show post in topic
Related topics
Topic
Replies
Views
Activity
Outputs change if re-using KVCache (past_key_values) for model.forward and generation
🤗Transformers
5
408
January 22, 2025
Model.generate use_cache=True generates different results than use_cache=False
Intermediate
3
424
March 4, 2025
Pass CausalLM KV cache into the next inference batch
🤗Transformers
0
585
October 14, 2023
How to cache common instruction prompt
🤗Transformers
16
2971
October 31, 2024
What does the `use_cache` in `generate` actually do?
🤗Transformers
1
2574
May 9, 2024