Difference between AutoModel and AutoModelForLM

Beginners
1

What is the difference between downloading a model using AutoModel.from_pretrained(model_name) and AutoModelForLM.from_pretrained(model_name) ?

1 Like
2

The first one will give you the bare pretrained model, while the second one will have a head attached to do language modeling. Note that AutoModelForLM is deprecated, you should use AutoModelForCausalLM, AutoModelForMaskedLM or AutoModelForSeq2SeqLM depending on the task at hand.

1 Like
3

Thank You.