Issues with Pythia model finetuning

geekyrahul · July 4, 2023, 7:38am

Hi,

I have been trying to fine tune the pythia 70m model on StackExchange dataset but it’s giving these errors:

   IndexError: Invalid key: 721 is out of bounds for size 0

Can you please guide me a bit to understand where I am making mistake?

gist.github.com

https://gist.github.com/rpalsaxena/762909fae03f421f5ac87c9193083055

pythia.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/gist/rpalsaxena/762909fae03f421f5ac87c9193083055/pythia.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"

This file has been truncated. show original

jchataigne · October 3, 2023, 7:51am

I cannot see your code but I got a similar error. I found the solution here:
you need to add a TrainingArguments to your Trainer, with remove_unused_columns=False (see the link for details)

Topic		Replies	Views
IndexError: Invalid key: 0 is out of bounds for size 0 🤗Datasets	0	200	April 4, 2024
IndexError: Invalid key: 16 is out of bounds for size 0 🤗Datasets	24	19018	November 19, 2023
[LayoutLMv3] index out of range in self inside outputs = model(**encoding) Models	3	2459	February 21, 2024
IndexError: string index out of range on TAPAS fine-tuning Beginners	4	1282	February 18, 2021
Invalid Key Error when Training GPT2 from Scratch using trainer.train() 🤗Transformers	3	1323	April 15, 2024

Issues with Pythia model finetuning

Related Topics