Re-Training with new number of classes

🤗Transformers
1

Hi All.

I already trained an NER (token classification) model on a custom training dataset with 19 classes. You can explore it here marefa-nlp/marefa-ner

The base model which I used to fine-tune my model was xlm-roberta-large

I have now a new dataset and need to use the last trained model marefa-nlp/marefa-ner as the base model this time.

The problem is that the last model was trained to predict the class out of 19 classes, while the new dataset is designed for just 6 classes.

I tried to load the model and reset the configurations to the xlm-roberta-large configuration like this

from transformers import AutoModelForTokenClassification, AutoConfig

base_model = "xlm-roberta-large"
ft_model = "marefa-nlp/marefa-ner"

config = AutoConfig.from_pretrained(base_model)

ner_model = AutoModelForTokenClassification.from_pretrained(ft_model, num_labels=19)
ner_model.config = config 

# THEN using the ner_model to train with the new dataset

but seems not working, as it still requires that the head size be 19

===

Does anyone know how to solve this?

Thanks

2

Hi,

This can be done by passing the additional argument ignore_mismatched_sizes=True to the from_pretrained method.

3

Thank you @nielsr . It works now.
I think this must be highlighted more in the new documentation.