Getting error - trainer.train()

🤗Transformers
1

Hello, I am trying to fine tune a GPT2 model with my data. While i am use train() method getting below error.

KeyError: ‘Invalid key. Only three types of key are available: (1) string, (2) integers for backend Encoding, and (3) slices for data subsetting.’

below is my code

cleaned_text =“”

from transformers import GPT2Tokenizer

tokenizer = GPT2Tokenizer.from_pretrained(‘gpt2’)

Add a new padding token to the tokenizer

tokenizer.add_special_tokens({‘pad_token’: ‘[PAD]’})

Tokenize the text using GPT-2 tokenizer

tokenized_text = tokenizer(cleaned_text, truncation=True, padding=True)

training_args = TrainingArguments(
output_dir=“./gpt2-finetuned”,
overwrite_output_dir=True,
num_train_epochs=3,
per_device_train_batch_size=2,
save_steps=10_000,
save_total_limit=2,
)

trainer = Trainer(
model=model,
args=training_args,
train_dataset=tokenized_text,

tokenizer=tokenizer,

)

trainer.train()

2

hello, how were you able to fix this issue?

3

any updates on this please ?
i’m facing the same problem when fine-tuning mistral 7b

4

Could you provide the full error log?