Hi,
Is the max_model_input_sizes
field on a tokenizer used anywhere, and how does it interact with the max_position_embeddings
field on a model? I don’t see any usages of this field outside of a research script.
Thanks,
Norman
Hi,
Is the max_model_input_sizes
field on a tokenizer used anywhere, and how does it interact with the max_position_embeddings
field on a model? I don’t see any usages of this field outside of a research script.
Thanks,
Norman