ESM Fast Tokenizer

nina789 January 15, 2024, 9:10pm 1

Is there a FastTokenizer implementation for ESM models?

Topic		Replies	Views
BertTokenizerFast for stsb-xlm-r-multilingual model 🤗Tokenizers	3	602	April 8, 2021
What is required to create a fast tokenizer? For example for a Marian model 🤗Tokenizers	0	261	March 16, 2023
Fast tokenizer for marianMTModel 🤗Tokenizers	1	412	September 26, 2022
SentencePiece user_defined_symbols and fast tokenizers 🤗Tokenizers	1	1008	January 3, 2024
How to convert Tokenizer to TokenizerFast? Beginners	1	451	September 30, 2020