[Dataset] Efficient LLM papers (quantization, LoRA, MoE, FlashAttention) from arXiv + Semantic Scholar — 1,734 records, quality-scored, JSONL

Show and Tell
1

Most of us aren’t training frontier models — we’re trying to fit a good one onto the
hardware we actually have. The research that makes that possible (quantization, LoRA/PEFT,
mixture-of-experts, FlashAttention, KV-cache tricks, Mamba/SSMs) is scattered across
hundreds of arXiv papers, and it’s some of the fastest-moving work in ML right now.

So I assembled it into one dataset: fineset-io/efficient-llm-papers

  • 1,734 papers on efficient LLM inference & training, published 2022–2026
  • arXiv (cs.LG, cs.CL) cross-referenced with Semantic Scholar, duplicates merged
  • Each record has a citation-normalized quality_score (0–1) so you can pull the
    high-impact methods and skip the noise
  • Plain JSONL: title, abstract, authors, published_date, citation_count,
    has_code/code_url, quality_score

I find it useful as a “what’s the current state of the art for making this cheaper”
reference — and as a clean corpus if you’re fine-tuning a model to reason about
efficiency techniques.

Happy to take suggestions on gaps or answer questions about how the pipeline works.

1 Like