Typical sampling decoding technique

[Gozdi]

Hi I recently came across new decoding technique typical sampling

Their github repo (GitHub - cimeister/typical-sampling: 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.) is forked from oryginal transformers repo as i guess it is already implemented but i don’t know how to use it. Can somone explain how to use this in transformers library? Thanks

[judisi]

Hey @Gozdi , I am currently having the same question. Did you maybe find a solution in the meantime? Greetings