Issues with Pythia model finetuning

geekyrahul · July 4, 2023, 7:38am

Hi,

I have been trying to fine tune the pythia 70m model on StackExchange dataset but it’s giving these errors:

   IndexError: Invalid key: 721 is out of bounds for size 0

Can you please guide me a bit to understand where I am making mistake?

gist.github.com

https://gist.github.com/rpalsaxena/762909fae03f421f5ac87c9193083055

pythia.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/gist/rpalsaxena/762909fae03f421f5ac87c9193083055/pythia.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"

This file has been truncated. show original

jchataigne · October 3, 2023, 7:51am

I cannot see your code but I got a similar error. I found the solution here:
you need to add a TrainingArguments to your Trainer, with remove_unused_columns=False (see the link for details)

Topic		Replies	Views
IndexError: Invalid key: 0 is out of bounds for size 0 🤗Datasets	0	532	April 4, 2024
Error in fine-tuning bert with custom text Beginners	0	402	April 14, 2022
Segformer fine-tuning: error with the metrics Beginners	7	1121	October 31, 2022
IndexError: Invalid key: 16 is out of bounds for size 0 🤗Datasets	26	22953	June 5, 2024
Index Error: Target {} is out of bounds 🤗Transformers	0	243	May 13, 2024

Issues with Pythia model finetuning

Related topics