Unable to clone repository via ssh

I followed Git over SSH to test my SSH authentication

(base) ➜  ssh -T git@hf.co
Hi ivnle, welcome to Hugging Face.

ivnle is the correct account that I added my public key to, but I’m unable to clone this repository.

(base) ➜  git clone hf.co:datasets/bigcode/starcoderdata data/starcoderdata-raw
Cloning into 'data/starcoderdata-raw'...
ivanlee@hf.co: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

I’ve obserbed the same issue across multiple machines and repositories

(ctt) (base) ➜  git clone hf.co:datasets/karpathy/tiny_shakespeare
Cloning into 'tiny_shakespeare'...
ivanlee@hf.co: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

I can use https to clone

(ctt) (base) ➜  git clone https://huggingface.co/datasets/karpathy/tiny_shakespeare
Cloning into 'tiny_shakespeare'...
remote: Enumerating objects: 36, done.
remote: Counting objects: 100% (2/2), done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 36 (delta 0), reused 0 (delta 0), pack-reused 34
Unpacking objects: 100% (36/36), 8.49 KiB | 115.00 KiB/s, done.

but not all repositories allow for https clone e.g. starcoderdata