Small Decoder-only model < 1B parameters

Models
1

Are there any decoder-only llama, mistral, gemma or otherwise that has < 1B parameters?

Any recommendations, esp. ones that are good at multilingual tasks?

2

Also asked on Reddit

3

From the open leaderboard, there’s a few that have relatively low MMLU-PRO scores but interestingly, any model < 1B params didn’t beat the flan-t5-large model.

image
image1920×792 89.6 KB