[Paper] Delta Compression: Towards Efficient Semantic Compression

Research
1

Hello everyone,

I would like to share my recent paper and Proof of Concept on a method I call “Delta Compression.”

Abstract

We propose “Delta Compression,” a novel approach to information compression designed to overcome the context length limitations of Large Language Models (LLMs). This paradigm represents information not as static data, but as a dynamic “parameter delta” that functionally alters a shared, deterministic base model. It utilizes a hypernetwork encoder to transform input information into a PEFT-style delta, enabling lossless retrieval of complex information from a compact, prompt-addressable representation.

PoC Summary

In our proof-of-concept, the following was observed:

  • 70 distinct Japanese texts (totaling approx. 70,000 tokens, 416KB as uncompressed files) were encoded into a minimal LoRA delta.
  • The resulting delta file has a size of approx. 396KB (rank 1, float16).
  • It was confirmed that the original information could be losslessly restored from this delta in response to individual prompts.

Links

Feedback & Discussion

Any thoughts or feedback on this research would be greatly appreciated.

Regarding arXiv Endorsement

I am an independent researcher seeking an endorsement to submit this paper to arXiv. For those who are eligible and willing to endorse, here is the formal request generated by arXiv:

Kazunori Kondo requests your endorsement to submit an article to the cs.CL section of arXiv. To tell us that you would (or would not) like to endorse this person, please visit the following URL:

arXiv user login

If that URL does not work for you, please visit

arXiv user login

and enter the following six-digit alphanumeric string:

Endorsement Code: ZNILHY

Thank you for your time and consideration.

1 Like
2

Welcome and congratulations on your first post @Mocha7195
I read as much as I can here and data compression is a term that catches my eye.
I am new here and learning so a welcome is what I have for you today.

EDIT: Very professional. As I say I am just learning but thank you for your work.

2 Likes
3

Thank you so much for the warm welcome and your very kind words! I’m glad the topic of data compression caught your eye. I’m new here as well, so I’m looking forward to reading other posts and learning from everyone.

1 Like