GGUF on serverless API?

myonlyeye April 13, 2024, 7:49am 1

Newbie question - is it possible to run a GGUF with the serverless API and if so, how do I configure the config.json to make it work?

Topic		Replies	Views
Gen AI on GCP GKE Models	2	350	January 16, 2024
Time out on all API model Models	4	238	December 12, 2023
Question from a new user Beginners	0	135	January 27, 2023
Sagemaker Serverless Inference Amazon SageMaker	21	7738	December 22, 2022
How can I use "Hosted inference API"? Beginners	0	238	May 7, 2023