Lets say I have a dictionary containing some IDs and tokens corresponding to those ids. Example: {‘ID1’: 1, ‘ID2’: 2, ‘ID3’: 3}. What is the best way to create a tokenizer from this python dictionary ?
1 Like