How hard is it to finetune an ELECTRA model for multihead regression & classification?


I intend to first pretrain my own ELECTRA model from scratch and then finetune it on a smaller set of labeled samples.

The thing is, the task I’m solving requires extracting multiple real-values (regression) and several class labels for independent classes. What steps do I have to take to get there? Are there any resources that might be of help to me?

Thanks a bunch!