Hi,
I have my main.py file which downloads a specific object from ShapeNetCore dataset if the download doesnt exist already in my local specified directory.
The problem that I have is that once I have it downloaded how do I load it from my local directory instead of downloading from huggingface?
dataset_dir = 'downloaded_data'
if os.path.exists(dataset_dir):
#Load from existing directory
dataset = load_dataset("ShapeNet/ShapeNetCore", data_dir=dataset_dir)
print("dataset loaded from local: ", dataset_dir)
else:
# Download the dataset from hugging face
dataset = load_dataset("ShapeNet/ShapeNetCore", data_files="02843684.zip", cache_dir=dataset_dir, download_mode="force_redownload")
print("Dataset downloaded from hugging face and saved to: ", dataset_dir)
# Print information about the downloaded dataset
print(dataset)
When the ShapeNetCore Dataset gets downloaded it is saved in this structure:
How should I fill out the arguments of load_dataset when wanting to use the already downloaded dataset?
Thank you