How are the inputs tokenized when model deployment?

I just noticed you have a spelling mistake in truncationtrucation is wrong

2 Likes