Wav2vec2 finetuning custom dataset

[Shiro]

Hello,

Thank you for sharing such nice model again on this framework.

I am trying to finetune a wav2vec2 model on a custom dataset (so not from the dataset package of huggingface). I have tried to follow these two tutorials :

• Fine-Tune Wav2Vec2 for English ASR in Hugging Face with 🤗 Transformers
• Fine-tuning with custom datasets — transformers 4.7.0 documentation
  but I did not find how to use multiprocessing when using trainer on a custom dataset ? Should I use dataloader class from torch ? For now I am using the regular Dataset class from torch.

I also encoutered memory issue with the GPU (16 gb) with a base Wav2vec2 model, even with batch size = 1. What is the maximum batch size for a base and large model for 16 gb, and with what length of sample ? (using fp16).

I thank you for the help

[theainerd]

You can convert your custom dataset if its a dataframe into Hugging Face format using this.

from datasets import Dataset, load_metric

train_data = Dataset.from_pandas(train_df)
test_data = Dataset.from_pandas(test_df)

If you are facing memory related problem, set num_proc = 1
It solved my problem.