thanks @wvangils - i will try this if i run into this again. I actually was able to work around the issue by just making a new notebook with the default python image in vertex and then installing huggingface and pytorch myself.
i noticed that if i make a new workbook NumPy/SciPy/scikit-learn 4 vCPUs, 15 GB RAM , NVIDIA Tesla T4 x (instead of the official pytorch one from the dropdown) and install pytorch myself with conda install pytorch torchvision torchaudio cudatoolkit=10.2 -c pytorch it all works.
So i’m guessing is some strange vertex bug with that image perhaps - i created a bug here for gcp so will see if someone there can re-create.
https://issuetracker.google.com/issues/243267023
Just linking all in here in case anyone else hits this.