Hey @ghadeermobasher there are several strategies that you can use to merge the entities and my suggestion would be to inspect the implementation in the token-classification pipeline. As you can see in the docstring there are quite some subtleties associated with merging entities correctly (depends on the language, tokenizer etc)!
1 Like