A Call for Expert Help: Building a Native Windows AI Wrapper to Empower My Students with Learning Disabilities

Beginners
1

I am a 78-year-old educator who mentors 9th grade students with significant learning disabilities. Recently, I have been wanting to develop a wrapper for small (1–2B parameter) GGUF and Llamafile models to run natively on Windows 10/11 systems. My objective is to configure the models using external weights (YAML, JSON, or INI files) that enable me to experiment with different sets of weights that, hopefully, lead to the generation of educational content that is more understandable and tailored to the cognitive ability weaknesses of my students. I plan to install the models and external weight files in the same subdirectory on solid state flash drives that have sufficient R/W speeds to support this effort.

I can’t enable WSL or install a Linux distribution on my student’s home computers because I fear that doing so may interfere with their essential assistive technologies installed on their computers.

I am humbly seeking guidance on how to make GGUF and llamafile models native to Windows that can have their weights easily changed by replacing an external file containing the weights. I take detailed notes, will value your time, and am willing to offer compensation for your expertise.

If you are interested in helping me help my students, please reach out to me via email at stevenijacobs49@gmail.com

Thank you for your consideration.

Steve Jacobs

1 Like
2

I don’t know if it fits this use case, but I think that Ollama can run any GGUF LLM relatively easily and quickly natively on Windows (Mac, Linux).

3

If you’re not referring to hyperparameters, such as temperature, this approach(tuning weights) might not be best way. Instead, you should create or gather a dataset to properly fine-tune the model(parameters/weights).

1 Like