I would like to add a special token (like, a CLS token in a model which was not pretrained with a CLS token) and finetune it to learn representations of that token. How should I go about this?
I would like to add a special token (like, a CLS token in a model which was not pretrained with a CLS token) and finetune it to learn representations of that token. How should I go about this?