Different lm_head size and vocab_size

🤗Transformers
1

Hi i was playing around with facebook/opt-1.3b and i noticed that it sometimes generates tokens with values bigger than the tokenizer vocab_size.
Indeed running this code

from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "facebook/opt-1.3b"
tokenizer_name = "facebook/opt-1.3b"

model = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(tokenizer_name)

print(model.lm_head)
print(tokenizer.vocab_size)

it seems that the output size of the lm_head layer is bigger than the vocab_size

Linear(in_features=2048, out_features=50272, bias=False)
50265

Am i using opt-1.3b wrong? is this supposed to happen for some reason? shall i just ignore the logits after the 50265-th?

Edit:
this happens when using generate with this arguments

outputs = self.model.generate(
    tokens.unsqueeze(0),
    return_dict_in_generate=True,
    output_scores=True,
    max_new_tokens=1,
    max_length=None,
)