One way you could do it is by precomputing the last hidden states’ CLS token embedding for each of the text in your dataset and storing it in a numpy array. Then you could concatenate this array with your desired additional features to accomplish classification task.
On a side note, you may want to rescale your additional features to the scale of bert embeddings.