How can I manually remove the head of a checkpoint that was fine-tuned for a specific task, and then add my own head for another task?
Let’s say I want to use the
google/bigbird-pegasus-large-arxiv checkpoint that has been fine-tuned for summarization on arxiv dataset. I cannot load it using
AutoModelForTokenClassification as Big Bird Pegasus doesn’t have the
Can anyone provide a generalizable solution to this?
Thank you very much!