Language model for wav2vec2.0 decoding

@patrickvonplaten @Beau
Hi guys, I had also implemented a simple KenLM with beam search decoding for Wav2Vec2CTC using: GitHub - parlance/ctcdecode: PyTorch CTC Decoder bindings

You may find it useful

Here is the repo:


This was very helpful. Thanks for posting it.

Could you please share the code you used for distilling wav2vec2?

Hey guys,

I’ve done some benchmarking with the pyctcdecode library and I think works quite well actually in combination with transformers.

Here is a repo where you can find some comparisons between Wav2Vec2 + LM vs. Wav2Vec2 + no LM as well as all the necessary scripts to run the eval: GitHub - patrickvonplaten/Wav2Vec2_PyCTCDecode: Small repo describing how to use Hugging Face's Wav2Vec2 with PyCTCDecode