Hi everyone,
How can I manually remove the head of a checkpoint that was fine-tuned for a specific task, and then add my own head for another task?
Let’s say I want to use the google/bigbird-pegasus-large-arxiv checkpoint that has been fine-tuned for summarization on arxiv dataset. I cannot load it using AutoModelForTokenClassification as Big Bird Pegasus doesn’t have the TokenClassification class.
Can anyone provide a generalizable solution to this?
Thank you very much!