Whisperx : CUDA out of memory

Hi Everyone. I am relevantly new to Huggingface. I am using whisperx to transcribe some audio and analyze them via the timestamps. I used a loop to go through all the dataset and transcribe them and put them in the database. However, no matter how much I change the batch size , use torch cache clear or python garbage collection I still eventually encounter CUDA out of memory error.
Model : whisper-medium
GPU: NVidia GeForce RTX 3060 Ti
OS: windows 11