Hi,
I want to use constrained decoding in my inference pipeline, after reading the following blog post: Guiding Text Generation with Constrained Beam Search in 🤗 Transformers
I noticed that supplying multiple disjunctive (OR) constraints acts as a conjunction (AND).
While this is a neat feature, it also means that if I have a batch of examples for inference, the entire batch would receive the same complex constraint. It is demonstrated in the example in the blog post where the two outputs have “scared” and an inflection of “scream”.
However, I’m looking for a way to supply individual constraints to each example in the batch.
Is it possible?
I don’t want to generate each example individually, as it is inefficient on large datasets…