Where to look for a loss definition for a pretrained model?

Beginners
1

I am using facebook/opt-350m model:

model = AutoModelForCausalLM.from_pretrained("facebook/opt-350m")

As far as I understand, its default loss is the crossentropy loss. But how can I verify it, and where can I see the implementation details? Thank you.

2

Hi, you can find the details here in the modeling_opt.py file :slight_smile:

1 Like
3

Thanks a lot Steven! Just what I needed, and it also helped me to resolve another question I posted before. Thanks!

1 Like