[Dataset] Synthetic data generation papers (self-instruct, distillation, augmentation) — 738 records, quality-scored, JSONL

Show and Tell
1

The bottleneck in almost every fine-tuning project I’ve seen isn’t the model — it’s
the data. And the methods for generating it (self-instruct, distillation from a stronger
model, data augmentation, evol-instruct, and the surrounding “is synthetic data actually
good for you” debate) are a moving target across arXiv.

I pulled that literature into a single dataset: fineset-io/synthetic-data-papers

  • 738 papers on synthetic data generation, published 2022–2026
  • arXiv + Semantic Scholar, cross-referenced and deduplicated into one record per paper
  • citation-normalized quality_score (0–1) per record for filtering by impact
  • JSONL: title, abstract, authors, published_date, citation_count, has_code, quality_score

Fitting given the meta-point — if you’re building a data-generation pipeline, this is the
research that pipeline is built on, in a clean form you can actually load and search.

What other topics would be useful as a maintained dataset? Happy to point the pipeline
at suggestions.

1 Like