Using detr with custom backbone

Models
1

I would like to use detr model with a fine tunned resnet-50 backbone, but I can see how to configure it.

Somebody know how to do it ?

Thanks,
Paolo

2

Hi,

Yes this is now supported using the AutoBackbone class.

Here’s how to do that:

from transformers import DetrConfig, DetrForObjectDetection

config = DetrConfig(backbone="microsoft/resnet50", use_pretrained_backbone=True)

model = DetrForObjectDetection(config)

Also, here’s how to randomly initialize a DETR model with a custom ResNet model:

from transformers import DetrConfig, ResNetConfig, DetrForObjectDetection

backbone_config = ResNetConfig.from_pretrained("microsoft/resnet-50", out_indices=[0,1,2,3])

config = DetrConfig(backbone_config=backbone_config)

model = DetrForObjectDetection(config)
3

Hi Niels,
thank you for your answer.

Paolo

4

this will load the pretrained backbone “microsoft/resnet-50” with randomly initialized encoder, decoder, and head of the transformer, am i right?
but what i’m trying to do, but i’m unable to do it is, i need to use a pretrained transformer, for exmaple: “microsoft/conditional-detr-resnet-50”, but i want to change the backbone to a fine tunned resnet backbone that i have the .pth or ckpt file for it

1 Like