Change length of GPT-neo output

Any way to modify the length of the output text generated by the GPT-neo inference API?


Does anyone know a solution for this?

1 Like

I am also interested in this

1 Like

Hi! You just need to modify the “max_length” parameter in your Json body, when requesting via API.

1 Like

I know that, but how do you go beyond 500 tokens?

As far as I know, the max output through API is 256 tokens (~200 words).

Change max_length:
res = generator(prompt, max_length=70, do_sample=True, temperature=0.9)