I think @stas or @patrickvonplaten have more experience with Adafactor.
Note that it won’t stay in the library forever: merging it was overspreading ourselves a little bit too much in optimizers territory and we now realize we don’t have the manpower to properly maintain it. So you should use a version from another library to be future-proof 