I am trying to train distilgpt2 with a custom last layer for text generation using tensorflow. While training the error message I received is as follows :
`InvalidArgumentError: Exception encountered when calling layer ‘transformer’ (type TFGPT2MainLayer).
input_ids must be smaller than the embedding layer’s input dimension (got 50257 >= 50257)
Condition x < y did not hold.
First 3 elements of x:
[43669 1312 655]
First 1 elements of y:
[50257]
Call arguments received by layer ‘transformer’ (type TFGPT2MainLayer):
• input_ids=tf.Tensor(shape=(10, 511), dtype=int32)
• past_key_values=None
• attention_mask=tf.Tensor(shape=(10, 511), dtype=int32)
• token_type_ids=None
• position_ids=None
• head_mask=None
• inputs_embeds=None
• encoder_hidden_states=None
• encoder_attention_mask=None
• use_cache=True
• output_attentions=False
• output_hidden_states=False
• return_dict=True
• training=False`
I am using google colab and my transformer version are huggingface-hub-0.13.4 tokenizers-0.13.3 transformers-4.28.1. I am trying to figure out the error but I am a little confused when I see this (got 50257 >= 50257) and First 3 elements of x:
[43669 1312 655]
First 1 elements of y:
[50257]
It would be of immense help if someone can help me understand.