Training and evaluation loss goes down however, WER score stays the same

I have a *.ipynb file in which i am trying to fine-tune wav2vec2.0 XLSR large model on common voice Farsi dataset.
I don’t know why but as steps go further, loss decreases but WER score stays the same.
This is the link to my *.ipynb file.

Can anybody please help me with this?
Thank you