Train T5 decoder only on a different language

I would like to train the T5 decoder in a different language with a different tokenization scheme and take advantage of the pretrained encoder’s ability to encode English. Is it possible to do with the Huggingface library? Any pointers?

Thanks in advance!