Using the Trainer class with AutoModel

[FDSRashid]

I am following the tutorial from chapter 7 of the NLP course in order to fine tune a Bert model on my own dataset, for the same purpose of the tutorial: fine tuning a model for domain adaption. The code in the tutorial uses a AutoModelForMaskedLM object. I was wondering if its reasonable to use just the AutoModel class instead. My thinking is that i want to fine tune a model to get better embeddings for my dataset, not necessarily for any specific task. For clarification, the code I am following is here.