Grouping Tokens after Token Classification

Is there a way to group tokens after token classification via HF? I see something similar in Rasa. However, I am not sure it is the best way to do it as they are giving group numbers to the model to train on. However, If a document contains more groups than the documents in the training data, the RASA implementation fails.

I think I am looking for a solution like (kinda supervised) clustering which is independent of the number of groups in the documents.


The token classification pipeline has the ability to group tokens, as seen here.

The front-facing API is an “aggregation strategy”. See the docs for more info.