I built a small language model from scratch. Here's how it works

I’ve been building Orin AI (AE-Assistant), a small language-model project designed to run locally on ordinary laptop hardware without a dedicated GPU.

The project explores a decoder-only Transformer, custom BPE tokenization, and experiments with recurrent-depth architectures and retrieval-based grounding.

I made a video walking through the project and showing the model running.

YouTube breakdown:

I’m still developing it, and I’d appreciate constructive feedback from people interested in small language models and training models from scratch.

What would you improve first?

2 Likes