I came up with an idea on kv cache size that is lossless. I don’t know enough about how to implement it, so i’m looking for someone who can do it. It’s going to change the industry in a very big way, I know this because it’s already changed another industry in an earth shaking way, and it can be applied here as well.
1 Like