Hi everyone,
I am currently working with some LMs (GPT2, BERT and T5 like). For some experiments I would like to use a “predetermined” vocabulary. If I have the list of the words, is it possible to build a tokenizer manually setting its vocabulary with my vocabulary?
Thanks in advance!