I’m using HF spaces + docker with persistent disk storage, and using the HfApi to upload a folder of large data files:
hf_api = HfApi() dataset_path = os.path.join('data', 'datasets', d) hf_api.upload_folder( folder_path=os.path.abspath(dataset_path), path_in_repo=dataset_path, repo_id=hf_space, repo_type='space',)
Which seems to get uploaded correctly:
However when I try to read files from /data once the docker image boots up they don’t seem to exist, using os.listdir I see an empty ‘/data’ dir once the docker image boots up.
Is there something I’m misunderstanding?