Issues with Pythia model finetuning


I have been trying to fine tune the pythia 70m model on StackExchange dataset but it’s giving these errors:

   IndexError: Invalid key: 721 is out of bounds for size 0

Can you please guide me a bit to understand where I am making mistake?

I cannot see your code but I got a similar error. I found the solution here:
you need to add a TrainingArguments to your Trainer, with remove_unused_columns=False (see the link for details)