I need to create my own dataset based on mlabonne/orpo-dpo-mix-40k. but when i does it and create a dataset for ORPO training it gives error

🤗Datasets
1 
---------------------------------------------------------------------------
UndefinedError                            Traceback (most recent call last)
<ipython-input-14-09e2a2282d7c> in <cell line: 3>()
      1 tokenizer = AutoTokenizer.from_pretrained(base_model)
      2 
----> 3 dataset = dataset.map(
      4     format_chat_template,
      5 )

8 frames
/usr/local/lib/python3.10/dist-packages/jinja2/environment.py in handle_exception(self, source)
    934         from .debug import rewrite_traceback_stack
    935 
--> 936         raise rewrite_traceback_stack(source=source)
    937 
    938     def join_path(self, template: str, parent: str) -> str:

<template> in top-level template code()

UndefinedError: 'str object' has no attribute 'role'

when i did futher checking, and got dataset.features from ‘mlabonne/orpo-dpo-mix-40k’ dataset it showed as

{'source': Value(dtype='string', id=None),
 'chosen': [{'content': Value(dtype='string', id=None),
   'role': Value(dtype='string', id=None)}],
 'rejected': [{'content': Value(dtype='string', id=None),
   'role': Value(dtype='string', id=None)}],
 'prompt': Value(dtype='string', id=None)}

but my dataset has features :

{'chosen': Value(dtype='string', id=None),
 'rejected': Value(dtype='string', id=None),
 'prompt': Value(dtype='string', id=None)}

there is no role, how to add it. i am using a csv for creating a dataset using dataset.

from datasets import load_dataset
dataset = load_dataset(‘csv’, data_files=‘my_file.csv’)

this is the code that gives error


def format_chat_template(row):
    row["chosen"] = tokenizer.apply_chat_template(row["chosen"], tokenize=False)
    row["rejected"] = tokenizer.apply_chat_template(row["rejected"], tokenize=False)
    return row

dataset = dataset.map(
    format_chat_template,
    num_proc= os.cpu_count(),
)
dataset = dataset.train_test_split(test_size=0.01)

this is for Fine-tune Llama 3 with ORPO (huggingface.co)

2

I have the same issue, Does anyone know how to solve it?

3

I was getting the same error.

However, there are two things to know that can cause the error:

  • Firstly, you need to prepare your dataset, such that your chosen column of string-encoded lists. Each list has at least two dicrionaries.
    The first dictionary should contain the prompt on which the chosen answer should be answered by the assistant. The second dictionary in the same list should be the chosen answer itself, associated with the role key as the assistant.

For a better view, your chosen column should contain such structures:

[{'content': prompt, 'role: 'user'} , {'content': chosen_answer, 'role: 'assistant'}

You should proceed further with applying the same technique with the rejected answers, and store them in similar structures.

  • Secondly, after you are done with the data processing part and your dataset is ready, you should make sure you parse these lists back correctly, which can sometimes give back errors, by parsing I mean converting the lists from the string format to actual list of dictionaries. (refer to ast for this step.

If this method doesn’t work, please provide me with more context such as examples from the dataset used.

1 Like