How to check if image exists at image url?

🤗Transformers
1

So every time I try to train my model (google/vit-base-patch16-224-in21k) on my dataset for image classification, it always gives a FileNotFound error (see below).

image
image2356×922 218 KB

Is there a way to go through the entire ‘Photos’ column and delete/identify whichever rows have an image url that doesn’t exist? Is there a python/huggingface function for that?

2

You can simply skip it as follows:

def to_pillow(examples):
    urls = examples['Photo']
    images = []
    for url in urls:
      try:
          image = Image.open(requests.get(url, stream=True).raw)
          images.append(image)
      except:
          pass
    
    examples['image'] = images
    
    return examples

dataset = load_dataset("TheNoob3131/mosquito-data")
dataset = dataset.map(to_pillow, batched=True)