Text classification and generation from the same model


I’m looking for some advice on whether it is possible to fine-tune a model to both classify and generate text. I have a labelled dataset of text and have had success fine-tuning BERT and GTP-2 to predict the labels. I have also been able to successfully fine-tune multiple GPT-2 models to generate text of each label-type.

For various reasons, I would ideally like to make predictions and generate texts from the same fine-tuned model. Is this possible?

Any advice would be appreciated.