Left a model overtrain tonight - by about a dozen checkpoints or so 
I had every checkpoint set to push to HF, and can see the commit hash of the checkpoint I’d like to retrieve my .safetensors from.
However, unsure on if it’s actually possible - as it looks like the commit itself just stored the hash of the .safetensors, rather than the actual .safetensors file (understandably - it’s not small).
Is there any sort of retrieval or rollback process possible with this? Or is it a case of having to restart the training from scratch with the target checkpoint in mind?