Language model for wav2vec2.0 decoding

@patrickvonplaten @Beau
Hi guys, I had also implemented a simple KenLM with beam search decoding for Wav2Vec2CTC using: GitHub - parlance/ctcdecode: PyTorch CTC Decoder bindings

You may find it useful

Here is the repo:

3 Likes

This was very helpful. Thanks for posting it.

Could you please share the code you used for distilling wav2vec2?

Hey guys,

I’ve done some benchmarking with the pyctcdecode library and I think works quite well actually in combination with transformers.

Here is a repo where you can find some comparisons between Wav2Vec2 + LM vs. Wav2Vec2 + no LM as well as all the necessary scripts to run the eval: GitHub - patrickvonplaten/Wav2Vec2_PyCTCDecode: Small repo describing how to use Hugging Face's Wav2Vec2 with PyCTCDecode

2 Likes