i have tried to use BART in the fill-mask pipeline to predict masked tokens, but the output sometimes might be more than one word, and the pipeline does not have an option for that.
In the documentation, I found out this link to mask-filling using BART
from transformers import BartForConditionalGeneration, BartTokenizer model = BartForConditionalGeneration.from_pretrained("facebook/bart-large", forced_bos_token_id=0) tok = BartTokenizer.from_pretrained("facebook/bart-large") example_english_phrase = "UN Chief Says There Is No <mask> in Syria" batch = tok(example_english_phrase, return_tensors="pt") generated_ids = model.generate(batch["input_ids"]) assert tok.batch_decode(generated_ids, skip_special_tokens=True) == [ "UN Chief Says There Is No Plan to Stop Chemical Weapons in Syria" ]
My question is, how to use this to obtain the top 5 preditions. in fill-mask pipeline, this would be equivalent to
unmasker = pipeline("fill-mask", model=model_name, tokenizer=tokenizer, top_k=10)
The final output I am hoping to obtain is:
["plan to stop the war", .. etc another 5 predictions ]