I guess I found the reason,
on deleting the previous checkpoint, it goes to the google drive bin and the bin does not delete it then (deletes after 30 days) and this results in occupied space.
@anon50011480 and @BramVanroy if you can verify this, we can
override the _rotate_checkpoint method of the Trainer to also clean the drive bin. That should resolve the issue