ModernBertForQuestionAnswering does not exist?

🤗Transformers
1

Is there a reason why the new ModernBERT does not support the question answering task?

1 Like
2

@tomaarsen Maybe you know?

1 Like
3

Hello!

The reason why ModernBERT might not support the question answering task could be due to its design and intended use cases. Many newer BERT variants, including ModernBERT, are often optimized for specific tasks or have been fine-tuned for better performance on tasks like classification, summarization, or token tagging.

Question answering (QA) typically requires a model to be fine-tuned on datasets specifically labeled for QA tasks (like SQuAD, etc.), which might not have been the primary focus of ModernBERT’s training. If you’re interested in using ModernBERT for QA, you could potentially fine-tune it on a QA dataset yourself or explore other BERT variants like RoBERTa or DistilBERT, which have strong QA capabilities due to their specific training and optimization for such tasks.

I hope that helps clarify things!

1 Like
4

Hi!

I’m talking about finetuning it on a QA dataset :slight_smile:

But the class ModernBertForQuestionAnswering, required for finetuning, simply does not exist.

2 Likes