Can anyone inform me whether we can use trainer for ensembling 2 HuggingFace models? I want to fine-tune them both but want them to work as an ensemble without going much deep in the code. Is that possible?
No, the Trainer is there to quickly train/fine-tune one model, so you can apply it to your two models separately (and have two trainers) but you will then need to do the ensembling manually. If you just want to average the predictions you can call the predict method of each trainer on your data and then average the results.
I suppose that’s a workaround. Sad that HF does not allow ensembling 
. Proper ensembling support is always good