I have fine-tuned an AutoModelForSequenceClassification
for binary classification.
I am now getting some more data. Quite little data. The data is of the same type. Binary classification. But it is specific for a customer.
I want to fine-tune on this data. But I don’t want to forget what I learnt - catastrophic forgetting.
Should I:
- Fine-tune with this data using a small learning rate?
- Or should I use adapters for this task? AdapterHub (AdapterHub) Remember that I still want to be able to do binary classification in general with what I learnt before.