Setting max_length does not limit length of output

>>> generator = pipeline("text-generation", model="HuggingFaceTB/SmolLM2-360M")
config.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 689/689 [00:00<00:00, 415kB/s]
model.safetensors: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 724M/724M [00:09<00:00, 73.1MB/s]
generation_config.json: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 111/111 [00:00<00:00, 697kB/s]
tokenizer_config.json: 3.66kB [00:00, 10.4MB/s]
vocab.json: 801kB [00:00, 9.48MB/s]
merges.txt: 466kB [00:00, 36.9MB/s]
tokenizer.json: 2.10MB [00:00, 53.9MB/s]
special_tokens_map.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 831/831 [00:00<00:00, 1.66MB/s]
Device set to use mps:0
>>> generator("I'm not sure if I know how to", max_length=50, num_return_sequences=3,)
Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.
Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.
Both `max_new_tokens` (=256) and `max_length`(=50) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
[{'generated_text': "I'm not sure if I know how to explain this. The problem basically is that you can't have a value of 0 in the output. I'm trying to do the following:\n\nfloat x = 2.0;\nfloat y = 0.0;\nfloat z = 1.0;\nfloat z2;\n\nz2 = z + x*y;\n\nI understand that y*z should be 2.0*0.0 = 0.0, but I'm not sure how to get the 0.0 in the z2 variable.\n\n## Answers\n\n0\n1. If you are trying to get the 0.0 in z2, please look at the following code:\nbool true = (z2*z2) > 0;\n\n// The result is 0.0\n\nfloat z2 = z2*z2;\n\n// The result is 0.0\n\nfloat z2 = z2*z2*z2;\n\n// The result is 0.0\n\n## Re: How to get 0 in a value in the output in a function\n\nThanks for the reply! I understand the problem now.\n\nI was trying"}, {'generated_text': "I'm not sure if I know how to do that.\n\nHow can I find the derivative of 1/x?\n\nI can't find the derivative of x^3\n\nI can't find the derivative of x^1/2\n\nI can't find the derivative of x^1/3\n\nI can't find the derivative of x^1/4\n\nI can't find the derivative of x^1/5\n\nI can't find the derivative of x^1/6\n\nI can't find the derivative of x^1/7\n\nI can't find the derivative of x^1/8\n\nI can't find the derivative of x^1/9\n\nI can't find the derivative of x^10\n\nI can't find the derivative of x^11\n\nI can't find the derivative of x^12\n\nI can't find the derivative of x^13\n\nI can't find the derivative of x^14\n\nI can't find the derivative of x^15\n\nI can't find the derivative of x^16\n\nI can't find the derivative of x^17\n\nI can't find the derivative of x^"}, {'generated_text': "I'm not sure if I know how to do this, but I tried to make a function that generates the 64 bit numbers and I got 128 bit numbers.\n\n```function rand64(digits = 128) {\nconst digits = digits;\nconst d = 7;\nconst s = 2147483647;\nconst e = -2147483648;\nconst f = 1;\nconst g = 2;\nconst h = 3;\nconst i = 4;\n\nconst m = 1024;\nconst d1 = 1 << d;\nconst d2 = 1 << d - d1;\nconst d3 = 1 << d - d1 - d2;\nconst d4 = 1 << d - d1 - d2 - d3;\nconst d5 = 1 << d - d1 - d2 - d3 - d4;\nconst d6 = 1 << d - d1 - d2 - d3 - d4 - d5;\nconst d7 = 1 << d - d1 - d2 - d3 - d4 - d"}]

It doesn’t seem like the max_length is being honored when this is run. This is straight out of the LLM course under the “Transformers, what can they do?” section.

1 Like

With the current Transformers library code, max_new_tokens takes precedence over max_length, so specifying max_new_tokens is the simplest approach.

1 Like

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.