Hello.
Recently I’ve been practicing sequence classification with BERT language models via transformers library.
I’ve noticed that BERT classifies with high score phrases like “asdfgh” to labels. I’ve been able to verify that unwanted(in just that case) feature on many different models trained with different datasets(plenty with examples and trained on many epochs).
I am pretty sure there are people just like me who might have wanted to kind of curb classification scores when it comes to non-word-like-phrases.
Yet I haven’t come up with complete solution. I’ve tried using spellcheckers, regular expressions and utilising BERT subword splitting, yet these help in that matter only partially.
Does anyone have had interest in simmilar case?
Were you able to deal it with somehow? If, yes could you tell me what approach have you undertook?
Thank you.