New data on same task - fine-tuning or adapter?

🤗Transformers
1

I have fine-tuned an AutoModelForSequenceClassification for binary classification.

I am now getting some more data. Quite little data. The data is of the same type. Binary classification. But it is specific for a customer.

I want to fine-tune on this data. But I don’t want to forget what I learnt - catastrophic forgetting.

Should I:

  • Fine-tune with this data using a small learning rate?
  • Or should I use adapters for this task? AdapterHub (AdapterHub) Remember that I still want to be able to do binary classification in general with what I learnt before.
1 Like