I have been experimenting with smaller open-weight models lately, and I keep wondering where the practical limit is.
With models getting better at tool calling, reasoning and agentic workflows, is it still worth running a larger model locally, or are smaller 2B–7B models becoming good enough for most personal workflows?
For people actually running models locally:
What model are you using?
What hardware are you running it on?
What task surprised you with how well a small model performed?
Would love to see some real-world setups rather than benchmark numbers.
I added a screenshot of one of the setups I was looking at here: