Fine-tuning Wav2v2.0: Loss increasing, WER decreasing

I’m fine-tuning the XLS-R large on my own dataset. After 20 epochs, the validation loss keeps increasing while validation WER keeps decreasing? Is this a sign of overfitting?