Availability of checkpoints earlier in training?

Hello! Thank you all, so much for everything you do for the NLP community, is has really changed my perception of how easy it is to just… do something, or try out an idea.

For a project I’m ideating (evaluating the learning dynamics of these large-scale language models), it would be quite helpful to have access to checkpoints of these models earlier in training. I’m assuming the publically available ones are of the converged models at the end of training – is there a chance or a place that checkpoints from the training process might be available for one (or more) of the models? A google search and peering through the documentation didn’t offer anything useful.

Thank you very much!