Hi everyone, I’m fine-tuning GPT-Neo on the GSM8K dataset and evaluating its performance using the lm-eval-harness . After training, I’ve noticed the model does a decent job generating reasoning steps, but still performs poorly on arithmetic accuracy — with correctness hovering around 2–3% on the te…

Fine tuning gpt-neo 2.7B with Lora on GSM8K - improve performance

suneetk55 June 9, 2025, 10:22pm 3

Ah thats smart, a system prompt should work well. thanks for the feedback!

2 Likes