I’m trying to use a model for token classification, but since the text comes from a DocX file, I want to add additional input which represents the formatting for each token. I’ve collected the formatting and created a (short) formatting vector per token which I concat to the LLM’s output vectors, but DataCollatorForTokenClassification doesn’t pad these additional vectors. I then get an error during training (e.g. “ValueError: expected sequence of length 1024 at dim 1 (got 882)”). What is the best/easiest way to override the DataCollator’s behavior to avoid this?