Missing dataset when following tutorials

Beginners
13

There is lot’s of JSON in .cache\huggingface\datasets\downloads\ like this:

4b7e15ecb751b44e2d66e32c3e7e02e4b37bb44e1abc932d9d00922ca665982a.c603ef2b6cf2bbebe908de7d918145ee128d003c6058bc6bab205a43c7dc18c0.py

4b7e15ecb751b44e2d66e32c3e7e02e4b37bb44e1abc932d9d00922ca665982a.c603ef2b6cf2bbebe908de7d918145ee128d003c6058bc6bab205a43c7dc18c0.py.json

79e27561b3c4da91e4249a0894b9a8408ed7508080d89f2c13a486a0ad29e7c2
79e27561b3c4da91e4249a0894b9a8408ed7508080d89f2c13a486a0ad29e7c2.json

a20af833c634c197b032a1337a371e7c741c0bc15c4a4ee719d51d9f448460fb
a20af833c634c197b032a1337a371e7c741c0bc15c4a4ee719d51d9f448460fb.json

a3dcca2a336ca35407db7364e360ef0bf50f9f2848a0b2230b9bde8448a4ea0a
a3dcca2a336ca35407db7364e360ef0bf50f9f2848a0b2230b9bde8448a4ea0a.json

afab7c85dc2a7cc7d0ab0bf2926d3c64cce46c5e4d05a1c3669515efcb2124bf.ee7442fd1e36f82b5f211a315d36608250a79865dd9b645657df2d770c4972b3
afab7c85dc2a7cc7d0ab0bf2926d3c64cce46c5e4d05a1c3669515efcb2124bf.ee7442fd1e36f82b5f211a315d36608250a79865dd9b645657df2d770c4972b3.json

d888d057846ead63005c95e175267409dd51a698218ebee0edf5ab216b133dea
d888d057846ead63005c95e175267409dd51a698218ebee0edf5ab216b133dea.json
extracted

fede8a3bc39f7c1f88af1b9eff20e181e04da5d9bd0fd76b428f218fe993c1d9.98f128f6a35a2e26f136376f175a86e07f91579870bd33f90a037b77f538ef5b
fede8a3bc39f7c1f88af1b9eff20e181e04da5d9bd0fd76b428f218fe993c1d9.98f128f6a35a2e26f136376f175a86e07f91579870bd33f90a037b77f538ef5b.json

Looks like they came in pair: the .json and the python source file you just linked. Which one should I change and rerun again to make sure all the files are downloaded correctly?

because I checked there’s only one extracted folder with tree like this

> C:.
> └───extracted
>     └───b557ce52f22c65030869d849d199d7b3fd5af18b335143729c717d29f6221baa
>         └───ADEChallengeData2016
>             ├───annotations
>             │   ├───training
>             │   └───validation
>             └───images
>                 ├───training
>                 └───validation

with missing images, do you think the code didn’t read all the .json and python code for downloading the datasets? Sorry this has become so messy, I really appreciate all the help I can get because I cannot find any reading about this. ADE20k documentation is also bad and I cannot download directly from their website until now

Btw is that Sagemaker yours or they have some codes to demonstrate segmentation somewhere?

1 Like