Hi,
Is the max_model_input_sizes field on a tokenizer used anywhere, and how does it interact with the max_position_embeddings field on a model? I don’t see any usages of this field outside of a research script.
Thanks,
Norman
Hi,
Is the max_model_input_sizes field on a tokenizer used anywhere, and how does it interact with the max_position_embeddings field on a model? I don’t see any usages of this field outside of a research script.
Thanks,
Norman