Hi there, I have been running a script to train a pretrained transformer on a summarization task. I am using custom data which I have put into my S3 bucket which is also the default bucket for this job. I have been getting this error in return, and have not been able to figure out what the solutio…

Running custom data files on run_summarization.py

OlivierCR June 22, 2021, 5:41pm 10

the error seems to be in here right?

datasets = load_dataset(extension, data_files=data_files, cache_dir=model_args.cache_dir)

is File "/opt/conda/lib/p the end of the error message?

Topic		Replies	Views
Using custom csv data with run_summarization.py in sagemaker Amazon SageMaker	4	2078	June 16, 2021
Data not passed to run_summarization.py Amazon SageMaker	8	2045	March 8, 2022
Sagemaker Text Summarization Fine Tuning Job failing Amazon SageMaker	6	1588	June 9, 2022
Sagemaker gpt-j train file error Amazon SageMaker	27	2912	February 22, 2024
Fine Tuning GPT-2 - Training job only using test sample size of 5 Amazon SageMaker	4	2148	February 6, 2023