Newbie question - is it possible to run a GGUF with the serverless API and if so, how do I configure the config.json to make it work?
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| What's the rate limit? | 0 | 336 | April 22, 2024 | |
| Serverless Inference API doesn't seem to support a dedicated JSON mode | 0 | 205 | June 23, 2024 | |
| Serverless Inference API error on new model | 5 | 308 | September 9, 2024 | |
| Serverless inference issues for a new Go library | 4 | 26 | March 18, 2025 | |
| Issue converting FLUX.1-dev model to GGUF format | 1 | 456 | September 12, 2024 |