Seeking Local AI Model for Assisting Students with Coding Exercises

Beginners
1

Hello everyone,

I’m a computer science teacher looking for a local AI model that can assist my students with their coding exercises. The ideal model would:

  • Allow students to submit their coding exercise along with their proposed solution.
  • Evaluate the solution’s correctness.
  • Identify any edge cases the student might have overlooked.
  • Provide a score for the solution.

I’m specifically seeking a solution that can run locally on our school server, allowing students to use it during lab sessions.

I’ve had success with the following prompt when using Claude:

** You’re a programming assistant helping students improve their coding skills. Here’s how to guide the conversation:

  1. Ask the student to provide the full question or exercise and their code solution.
  2. After the student provides the information, analyze their solution and give:
  • A score from 0 to 100 for the solution.
  • A brief hint on where the code could be improved or what to consider.
  1. If the student asks for another hint, provide additional guidance, but avoid giving the full solution.
  2. Allow the student to submit revised versions of the code as many times as they want. Each time, give a new score and another hint if needed.
  3. The hints should be small and incremental, encouraging the student to think for themselves.
  4. Never give the full solution at any point, even if the student asks for it.

When giving a score, deduct points as follows:

  • Minor syntax errors (like missing semicolons): slight deduction.
  • Logical issues or bugs: moderate deduction.
  • Completely incorrect solutions or misunderstanding of the problem: very low score or zero.

Encourage the student to keep trying and improving their code.

Remember, the goal is to help students learn and develop, not to solve the problem for them. **

This prompt works excellently with Claude, but the cost makes it impractical for regular use. When trying it with ChatGPT, the AI tends to provide the full solution instead of offering gradual hints, which is not ideal.

I’m looking for recommendations on local models or solutions that can replicate this functionality. The model should be able to analyze code, provide incremental hints, and avoid giving away complete solutions. Any suggestions or experiences with similar setups would be greatly appreciated!

2

Hi,

For coding, you might take a look at this benchmark: https://bigcode-bench.github.io/. It benchmarks various models for coding, both open- and closed-source. A leaderboard is hosted here: BigCodeBench Leaderboard - a Hugging Face Space by bigcode.

Currently, the best (but this might change in a week as models get shipped fast) include deepseek-ai/DeepSeek-Coder-V2-Instruct-0724 · Hugging Face and Qwen-2.5.

  • DeepSeek-V2 has 236 billion parameters, so it requires at least 236GB of GPU RAM in case you run it with 4-bit quantization.
  • Qwen-2.5 (specifically the 72B variant) is on top on livebench.ai for the “coding” section and is much more feasible to run in a local environment. The model collection includes GGUF quantized versions.

For smaller models, a good current model is the Qwen-2.5-Coder series, coming in both 1.5B and 7B sizes.

3

Thank you very much for the recommendation!
I don’t have very strong processing power, so I’ll try the smaller versions.
Can these models help guide students towards a solution without giving the full answer, and even provide a score?
Or is their focus more on code generation?

Also, do they support Hebrew, at least for responses? If not, I can use a translation tool to handle that.
thanks