Hi! I am currently fine-tuning a Wav2Vec2 model and I would like to know how I could modify training samples. In Tensorflow it’s possible to do this by calling dataset.map() but doing this with a datasets.arrow_dataset.Dataset will write a cache file. Besides that, it’s static and unnecessary. So…

Preprocessing & augmentation during training

mariosasko February 9, 2022, 4:14pm 2

Hi! I think Dataset.set_transform is what you are looking for.

Topic		Replies	Views
Dataset map preprocess throws ArrowInvalid 🤗Datasets	5	1303	September 16, 2021
Transform a tf.data.dataset to a datasets.dataset? Beginners	3	2390	September 30, 2022
Expanding an Audio Dataset with datasets.map()? Beginners	4	774	December 5, 2024
Use tf.data.Data with HuggingFace datasets 🤗Transformers	2	2641	March 22, 2021
Prepare func failed when mapped on audio dataset Beginners	0	331	July 16, 2022