Hugging Face API parameters

Beginners
1

Hi, complete noob here. I am trying to use the the inference api to do some text generation with the bloom model. When I get my generated text from the api in my code it seems to be using the “greedy” setting instead of the “sampling”. Like it is shown in the example here: bigscience/bloom · Hugging Face. So my question is how do I change the parameters(Detailed parameters) of the model because I guess that’s how I get the output to be more like “sampling” on the model webpage. Ive tried sending parameters like this:

const response = await fetch(
        "https://api-inference.huggingface.co/models/bigscience/bloom",
        {
            headers: { Authorization: `Bearer ${key}` },
            method: "POST",
            body: JSON.stringify({inputs: data, parameters: {top_k: 0, etc}})
        }
    );

But it does not work. Any help would be appreciated!

1 Like