Hybrid approach for text categorization (Rule based + ML)

I want to build a multi-label text categorization system.

For many of the categories keyword based logical rules exist with a low false positive and medium false negative rate, which means that the rules are quite strict and are missing some of the true labels.

I am looking for an approach that can leverage the advantages of the rule based approach, which are: high precision and no training data is needed.

On the other side I would like to combine it with the advantages of powerful language models like HuggingFace BERT, which are able to increase recall, by semantically extending the predefined rules.

The simplest approach would be 1) train ML model based on texts categorized by the rules (or k-nn) 2) combine the labels predicted by the model and the rules. But are there any other options for such a hybrid system?