Typical sampling decoding technique

Hi I recently came across new decoding technique typical sampling

Their github repo (GitHub - cimeister/typical-sampling: 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.) is forked from oryginal transformers repo as i guess it is already implemented but i don’t know how to use it. Can somone explain how to use this in transformers library? Thanks

1 Like