Can beam search be used with sampling?

🤗Transformers
1

Hello,

I am wondering if beam search can be used alongside sampling. I would like to sample from the language model to generate some text. However, in some cases, the generation does not look great. I was hoping to use beam search to return the most likely sampled generation.

Does the current version of Transformers allow using beam search with sampling? If that is not supported, what is the behavior of the model when using settings for both beam search and sampling together?

Thanks in advance for your help!
Artidoro

2

Hi Artidoro, welcome to the forum :hugs:

My guess is that beam search and sampling are compatible. That being said, nothing beats trying it out for yourself :slightly_smiling_face:

The way I’d try it is to first generate text with beam search and without sampling and after that generate text for the same prompt again, but with sampling switched on.

Because sampling makes the model non-deterministic you should get a different output once you switch on sampling, which indicates that tboth can be combined.

Hope that helps, keep us updated if you decide to try it out.

Cheers
Heiko

1 Like
3

Thanks a lot for your reply! I did try, as you suggested. Beam search + sampling leads to different generations with the same prompt.

1 Like