Subject: Request to register GKB-1000 as a Hugging Face Benchmark
Hello Hugging Face team,
I would like to request the registration of my dataset, General Knowledge Benchmark (GKB-1000), as an official benchmark on the Hugging Face Hub.
Dataset: Lelonthecodeur/General-knowledge-benchmark-for-llm at main
GKB-1000 contains 1,000 questions across five categories:
- Logical and mathematical reasoning
- Scientific and technical knowledge
- Language and communication
- Agentic and adaptive reasoning
- Critical thinking and reliability
I have added an eval.yaml file to the root of the repository, following the Hub’s evaluation specification.
Could you please review the submission and let me know the steps required to add GKB-1000 to the benchmark allow-list?
If there is any modification needed you can contact me .
Thank you.
Welcome! 
Process: you’ve done the main steps. Per the Hub docs, the eval.yaml is validated at push time, and the last step is getting in touch so HF staff can add the repo to the allow-list. Staff handle that, so tag @SaylorTwift in this thread or open a discussion on the dataset repo, as people have done for other benchmarks.
Before they review, I’d fix these:
- Near-duplicates. In the samples, MDQ-0494, 0495 and 0496 are identical except for the case ID. The algebra items (0001-0004) share the same template and the same answer text with different numbers. A “1,000 question” benchmark with this much templating has far fewer distinct questions. Either dedupe, or say clearly in the README how many unique families and variants there are.
- Split/config. Your
eval.yaml uses config: default and split: test. The repo has one .jsonl, so check that the README YAML declares it as the test split and that it loads with load_dataset(..., split="test").
- Two README files. There’s both
README.md and Readme.md. Keep one.
- Grader.
model_graded_fact with openai/o3-mini makes scores depend on a closed model. Document that choice and test the grader on a few known-good and known-wrong answers.
Hi @Lelonthecodeur, thanks for the request, and thanks @RDTvlokip for the thorough review!
I’m on the HF team handling the benchmark allow-list. For now we’re keeping official benchmarks to evals that are already widely used by the community, so we won’t add GKB-1000 to the allow-list yet. The eval.yaml and result files work as they are, so you can keep using them, and we’re happy to revisit once the benchmark has some adoption.
In the meantime, RDTvlokip’s points are worth fixing (deduplicating the templated questions, keeping one README, documenting the o3-mini grader). You can also drop evaluation_framework from your eval.yaml, it’s optional.
Ok thank you for the answer inwill fix everything and improve it