Huggingface implement new Optimizer

gordonhu May 25, 2023, 7:45am 1

Hi, we should incorporate the newly released Sophia optimizer into huggingface!

2 Likes

Topic		Replies	Views
Huggingface Optimizer 🤗Optimum	2	487	May 25, 2023
Trainer class optimization for transformer models Models	0	435	January 8, 2022
Transformers v3.0.0 is out! 🤗Transformers	0	1967	July 7, 2020
Pre-training with Lamb optimizer Research	7	4392	December 28, 2020
Transformers-cli - Python SDK 🤗Transformers	0	358	February 23, 2021