There are many ways to quantize model. GGML (5 quantization methods at least) GPTQ (4 quantization methods) and bitsandbytes(2 methods). So there are 11 models per each fine-tune. And there is uncountable number of fine-tunes.
So I propose to add ability to designate model as child model.
Because in my opinion currently model discovery is impossible.
How that would work. You go to llama 13b model page. There is a list of all daughter models. Alpaca Koala and other animals. Maybe even see a list of datasets per each model. They are most likely in original format without quantization. Then you go to alpaca page. There you see various quantized daughter models. You select it and download.
1 Like