NER - aggregation_strategy

I have custom trained a distilBERT NER model for extracting skills from Job Descriptions.

I am using aggregation_strategy = first.

There are my model results

The issues I am facing :

  1. The entity extraction fails, if the input text contains entities with space i.e., if we input ‘machine learning’ instead of ‘machinelearning’, the model ends up extracting only machine as a skill.

  2. Aggregation is grouping phrases in the sentence together.
    For example :

These issues persist with other aggregation strategies also i.e., simple/average/max

Any thoughts on why this is happening, and how to resolve this.

Thanks