Controlling bos, eos, etc in api-inference

Is there a way to control the beginning of sentence, end of sentence tokens through the inference api? I could not find it in the documentation.

Hi @cristivlad ,

Currently there is no way to override those within the API.
We are adding and end_sequence parameter to enable stopping the generation when using prompt-like generation (for GPT-Neo for instance).

For BOS, what did you want to do with it ?