Huggingface Optimizer

🤗Optimum
1

Hi, We should incorporate the newly released Sophia optimizer into huggingface !

2

Hi @gordonhu! Optimizers are managed in Transformers so I recommend that you open a feature request there :slight_smile:

3

Thank you for letting me know!