Background: I’m creating Keras models that are based on HF models such as - TFDistilBertForSequenceClassification (uncased).
I’m using distilbert as a feature tokenizer, and adding a classifier on top. It means I need to define inputs, use distilbert, bind the output and create a new Keras model.
Here’s the thing - when creating a model like that, I need to define a fix input length, for instance 512. Due to that, any sentence that will use this model, needs to be padded to 512. Even during inference time, which is a bummer because we are just wasting time. Any sentence of shorter length will cause an error.
When using HF implementation, I can use which ever input length I want, up to 512.
How did you achieve this feat?
I’m going over your repository: https://github.dev/huggingface/transformers/blob/main/src/transformers/models/distilbert/modeling_tf_distilbert.py
But still don’t get it. Please help