I was wondering if it is possible to finetune a BERT model on SQuAD, then delete the last x layers and then perform a SQuAD evaluation on the remaining layers? How would one go about doing this?
Would this help?
I am not exactly clear on what it does from reading the documentation but from my understanding, it only removes linear layers, not full BERT layers.
Any help would be greatly appreciated,