Text generation max length

🤗Hub
1

Hi. I’ve fine-tuned gpt2 model for text generation and trying to experiment with it in the hub ui. The problem is that it only generates short texts (much shorter in comparison to the original gpt2 model). When you send it a longer text, it emits an unknown error exception. I’ve tried to get more details about the error in the network tab of he browser, and that’s what I’ve seen:

{
    "error": "unknown error",
    "warnings": [
        "Input length of input_ids is 50, but `max_length` is set to 50. This can lead to unexpected behavior. You should consider increasing`max_new_tokens`.",
        "Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.",
        "There was an inference error: unknown error: indices[0] = [0, 50] does not index into shape [1,50] [Op:TensorScatterUpdate]"
    ]
}

So as I understood, somewhere max_length is set to 50, and I should increase it, but I can’t find where it is set. Initially I though that I should change property task_specific_params.text_generation.max_length in the config.json. However, I increased it from 50 to 500 and it had no effect.

As a result, the question is, where should I increase the max_length parameter to generate longer text in the widget?

2

It seems that I had to wait until the model reloads. After it happens it indeed starts to generate longer texts