Inference API for Tokenizers

Beginners
1

Hi everyone,

I’m trying to deploy my backend on Heroku and running into the 500 MB slug size limit because my code downloads two tokenizers from Huggingface.

Is there an API endpoint that I can ping to tokenize the same way you can do inference?

For reference, the two tokenizers are BertTokenizerFast.from_pretrained('bert-base-uncased') and SentenceTransformer( 'multi-qa-MiniLM-L6-cos-v1' )

Thanks!