Re-initialize decoder parameters of a pretrained model

What is the recommended way of re-initializing only the decoder weights of a pretrained model?
In the case of Mask2Former I noticed that it has a different weight initializing method depending on the layer.

from transformers import AutoModel

mask2former_model = AutoModel.from_pretrained("facebook/mask2former-swin-tiny-cityscapes-panoptic")

# re-initialize decoder weights...