How to add additional features last layer of T5 before finetuning on seq2seq

Beginners
1

I have a dataframe with:
<input_sentence, vec, label_sentence>
Vec is a vector of 5 floats (a PCA of additional data on each sample)
I want to finetune T5 to take input_sentence, add vec to the last hidden layer, and predict label_sentence.
How Can I do it?

Thanks!