Jointly train two-stage models using Trainer

Hi all,

I want to train a two-stage model containing model1 and model2, in which model2 takes model1’s result as input. For now, I could train model2 properly through Huggingface Seq2SeqTrainer, but I have no clue of how to jointly train model1 and model2 through the Huggingface Trainer. Could someone give me some advice? Thank you very much

I don’t think this is possible. For suc a use case, you should definitely check out Accelerate and use your custom training loop.