How to separate predictions map to create id2label.json

Beginners
1

Dear @John6666 and other community members

Since using self-made mask didn’t work last time, said I need ground-truth image for the model to know what I mean by river area, how should I separate the “river” from the rest of tensors to make the id2label.json? Or did I do it the other way around by training the masked image?

The example semantic_prediction[0]can spit out a tensor but I don’t think it means anything since matplotlib.pyplot.imshow(semantic_prediction[0]) can’t show anything

I’m currently making maps for each image, but so far the prediction are weirdly similar for different images, do their information differences negligible or should I do some normalization to make them seen?

1 Like
2

I’ve never tried segmentation before…
I collected some links that might be useful if you’re trying to do segmentation with Hugging Face.

3

Thanks I’ll check them out, sorry for keep asking you while you keep highlighting never done segmentation before, almost ran out of people to ask and I don’t know exactly what I’m trying to do

1 Like
4

No problem.:grinning: It’s also a learning experience for me. However, if you want to ask someone who knows a lot about AI models, it may be quicker to use HF Discord.
If you’re lucky, HF staff or someone knowledgeable will pass by the forum…

5

The thing is tagging you has been the most dependable results I had so far even compared to discord forum,

Read the forum, it’s exactly what I’m looking for, will try it first then ask again if I need some clarification; maybe there maybe here, thanks John.

1 Like
6

@John6666 Okay so I get these

image
image823×789 6.68 KB

and this

[{'score': None,
  'label': 'wall',
  'mask': <PIL.Image.Image image mode=L size=1080x1080>},
 {'score': None,
  'label': 'building',
  'mask': <PIL.Image.Image image mode=L size=1080x1080>},
 {'score': None,
  'label': 'sky',
  'mask': <PIL.Image.Image image mode=L size=1080x1080>},
 {'score': None,
  'label': 'tree',
  'mask': <PIL.Image.Image image mode=L size=1080x1080>},
 {'score': None,
  'label': 'mountain',
  'mask': <PIL.Image.Image image mode=L size=1080x1080>},
 {'score': None,
  'label': 'water',
  'mask': <PIL.Image.Image image mode=L size=1080x1080>},
 {'score': None,
  'label': 'fence',
  'mask': <PIL.Image.Image image mode=L size=1080x1080>},
 {'score': None,
  'label': 'rock',
  'mask': <PIL.Image.Image image mode=L size=1080x1080>},
 {'score': None,
  'label': 'signboard',
  'mask': <PIL.Image.Image image mode=L size=1080x1080>},
 {'score': None,
  'label': 'bridge',
  'mask': <PIL.Image.Image image mode=L size=1080x1080>},
 {'score': None,
  'label': 'boat',
  'mask': <PIL.Image.Image image mode=L size=1080x1080>},
 {'score': None,
  'label': 'pier',
  'mask': <PIL.Image.Image image mode=L size=1080x1080>}]

Is this what I move to here?


import datasets
import glob

IMAGES = glob.glob("D:\cropped_image_1080x1080\img_0")
SEG_MAPS = glob.glob("D:.\img_0.jpg")

dataset = datasets.Dataset.from_dict({
    "image": IMAGES,
    "label": SEG_MAPS
},
features=datasets.Features({
    "image":datasets.Image(),
    "label":datasets.Image()
                            })
)

I still don’t get how I move the fifth member of this pipeline that I got from here

from transformers import pipeline

semantic_segmentation = pipeline("image-segmentation", "shi-labs/oneformer_ade20k_swin_large")

to id2label, do I need to label it as implied in here?

1 Like
7

How do this work for 8725 images that might have different scores, I still don’t understand the concept behind id2label and pretrained model…

1 Like
8

Oh, it seems to have worked out well. No matter which model you use, it’s better to refer to the author’s model card rather than the automatically generated upper part of the HF.
For this model, it would look something like the link below.
The format required will vary depending on the model, but as long as you make the dataset in a way that is easy for you to use, that should be fine.
Uploading and downloading to HF’s dataset repo and organizing large amounts of data is the forte of the datasets library.
However, there may be a recommended dataset format for HF Trainer.

In the end, it’s the same thing that we need to write a few lines of Python before passing it to the model.:sweat_smile: I don’t think you need labels for use cases that don’t require labels. I don’t think you need labels for segmentation.

9

Then do I only need the mask for training? Every guide I found use id2label though, I understand that this is semantic segmentation so not only it is segmented but its also labelled as something

1 Like
10

Pipelines are convenient, but they perform complex processing internally, so they are like black boxes.
It might be better to try doing it manually at first. The method on the model explanation page is the most general-purpose method.

Then do I only need the mask for training?

Yes.

semantic segmentation so not only it is segmented but its also labelled as something

If that’s the case, you may need to label the data set…
It’s difficult to create data that doesn’t exist. Only a large AI or a human being could do it.

11

so I need to id2label for my mask, what should I do with each weight? Sorry I’m still confused and don’t know how to explain it tbh…

I know I need these weights for some fine tuning but I don’t know where to store it as. Or if I train my mask I got from applying the model to my picture is it what I need?

1 Like
12

Even if you have to create your own data in the end, semantic segmentation itself is an existing task.
I think there are already working datasets for semantic segmentation that you can use as samples. If you can imitate that structure, it will be the quickest.

13

It seems like you need a mask that uses multiple colors or shades to semantic segmentation. Your mask is black and white, 0 or 1, so it should be usable for two types of classification. I think it should be usable for whether or not it’s a river.

by json file like this (or opposite?):

{"0": "not river", "1": "river"}

With HF, JSON files that have been carelessly left lying around are sometimes configuration files with fixed file names…:sweat_smile:

1 Like
14

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.