Fine-tune DETR on custom dataset while keeping old class labels

Models
1

detr-resnet50 itself has 91 classes including things like person, bicyce, car, horse, etc.

I am fine-tuning DETR for 3 more classes on a custom dataset using this method that finetunes DETR for balloons:

Transformers-Tutorials/DETR/Fine_tuning_DetrForObjectDetection_on_custom_dataset_(balloon).ipynb at master · NielsRogge/Transformers-Tutorials · GitHub

However, we see that the only thing that the model can identify is balloon:

model = DetrForObjectDetection.from_pretrained("nielsr/detr-finetuned-balloon-v2", id2label={0:"balloon"})

I was wondering if there was any way that I could maintain the 91 old class labels and also my new class labels in the model so I do not need to load 2 separate models.

I am assuming that I need to load the pretrained model and train it on the old dataset + my new dataset.