T5 transformer and LSTM

Hello everyone, recently I have fine tuned T5 transformer for text summarization. The result is really great. Now I want to make a hybrid model using T5 and LSTM. Is it possible to make hybrid model? If possible how can I implement it?. Thanks in advance.