I have custom trained a distilBERT NER model for extracting skills from Job Descriptions.
I am using aggregation_strategy = first.
There are my model results
The issues I am facing :
The entity extraction fails, if the input text contains entities with space i.e., if we input ‘machine learning’ instead of ‘machinelearning’, the model ends up extracting only machine as a skill.
Aggregation is grouping phrases in the sentence together.
For example :
These issues persist with other aggregation strategies also i.e., simple/average/max
Any thoughts on why this is happening, and how to resolve this.