Idle GPU when finetuning whisper tiny

I had the same problem and after some experimentation, I found out it may be because the data is not fed enough to GPU and it starves, probably because the tiny model does not have many parameters and finishes the calculations quickly (RTX-3090 here).

Although it did not fill the GPU fully, setting the dataloader_num_workers to virtual core count (I have a 6/12 CPU, so I set it to 12) helped a lot. I had many other changes like sharding etc, but I think this is the main parameter that helped.

1 Like