Usage question: How to get binary mask for just 1 class? What class number represent what object?

[fatbringer]

问题确认 Search before asking

• 我已经搜索过问题，但是没有找到解答。I have searched the question and found no related answer.

请提出你的问题 Please ask your question

Hi ! I want to get a binary mask using the semantic segmentation to get regions containing people, and other regions
For example from these 2 pictures

Original picture:

Desired binary mask:

I notice that in the following code that i have attempted

## Actual inference performed here, get logit of every class every pixel
with paddle.no_grad():
    pred, logit = inference(  
        segmodel,
        data["img"],
        trans_info=data["trans_info"],
        is_slide=False,
        stride=None,
        cr

the logit variable contains 19 2D arrays that has the confidence of each class. However, i do not know which class is which class.
Is there a list that i can refer to in order to know what number is which class?

Thank you!

[Asthestarsfalll]

Which dataset do you use?
If you use Cityscapes, it should be：

 
 
                     name |  id | trainId |       category | categoryId | hasInstances | ignoreInEval|        color
 
    --------------------------------------------------------------------------------------------------
 
                unlabeled |   0 |     255 |           void |          0 |            0 |            1 |         (0, 0, 0)
 
              ego vehicle |   1 |     255 |           void |          0 |            0 |            1 |         (0, 0, 0)
 
     rectification border |   2 |     255 |           void |          0 |            0 |            1 |         (0, 0, 0)
 
               out of roi |   3 |     255 |           void |          0 |            0 |            1 |         (0, 0, 0)
 
                   static |   4 |     255 |           void |          0 |            0 |            1 |         (0, 0, 0)
 
                  dynamic |   5 |     255 |           void |          0 |            0 |            1 |      (111, 74, 0)
 
                   ground |   6 |     255 |           void |          0 |            0 |            1 |       (81, 0, 81)
 
                     road |   7 |       0 |           flat |          1 |            0 |            0 |    (128, 64, 128)
 
                 sidewalk |   8 |       1 |           flat |          1 |            0 |            0 |    (244, 35, 232)
 
                  parking |   9 |     255 |           flat |          1 |            0 |            1 |   (250, 170, 160)
 
               rail track |  10 |     255 |           flat |          1 |            0 |            1 |   (230, 150, 140)
 
                 building |  11 |       2 |   construction |          2 |            0 |            0 |      (70, 70, 70)
 
                     wall |  12 |       3 |   construction |          2 |            0 |            0 |   (102, 102, 156)
 
                    fence |  13 |       4 |   construction |          2 |            0 |            0 |   (190, 153, 153)
 
               guard rail |  14 |     255 |   construction |          2 |            0 |            1 |   (180, 165, 180)
 
                   bridge |  15 |     255 |   construction |          2 |            0 |            1 |   (150, 100, 100)
 
                   tunnel |  16 |     255 |   construction |          2 |            0 |            1 |    (150, 120, 90)
 
                     pole |  17 |       5 |         object |          3 |            0 |            0 |   (153, 153, 153)
 
                polegroup |  18 |     255 |         object |          3 |            0 |            1 |   (153, 153, 153)
 
            traffic light |  19 |       6 |         object |          3 |            0 |            0 |    (250, 170, 30)
 
             traffic sign |  20 |       7 |         object |          3 |            0 |            0 |     (220, 220, 0)
 
               vegetation |  21 |       8 |         nature |          4 |            0 |            0 |    (107, 142, 35)
 
                  terrain |  22 |       9 |         nature |          4 |            0 |            0 |   (152, 251, 152)
 
                      sky |  23 |      10 |            sky |          5 |            0 |            0 |    (70, 130, 180)
 
                   person |  24 |      11 |          human |          6 |            1 |            0 |     (220, 20, 60)
 
                    rider |  25 |      12 |          human |          6 |            1 |            0 |       (255, 0, 0)
 
                      car |  26 |      13 |        vehicle |          7 |            1 |            0 |       (0, 0, 142)
 
                    truck |  27 |      14 |        vehicle |          7 |            1 |            0 |        (0, 0, 70)
 
                      bus |  28 |      15 |        vehicle |          7 |            1 |            0 |      (0, 60, 100)
 
                  caravan |  29 |     255 |        vehicle |          7 |            1 |            1 |        (0, 0, 90)
 
                  trailer |  30 |     255 |        vehicle |          7 |            1 |            1 |       (0, 0, 110)
 
                    train |  31 |      16 |        vehicle |          7 |            1 |            0 |      (0, 80, 100)
 
               motorcycle |  32 |      17 |        vehicle |          7 |            1 |            0 |       (0, 0, 230)
 
                  bicycle |  33 |      18 |        vehicle |          7 |            1 |            0 |     (119, 11, 32)
 
            license plate |  -1 |      -1 |        vehicle |          7 |            0 |            1 |       (0, 0, 142)

only need to focus on trainId

[fatbringer]

@Asthestarsfalll im using a config file pp_liteseg_stdc2_cityscapes_1024x512_scale1.0_160k.yml and model-pp_liteseg_stdc2_cityscapes_1024x512_scale1.0_160k.pdparams. I wanted to use the pre-trained model and weights that were trained on cityscapes.

Are these the correct ones to use?

Ok i will look at the trainId. I found it was odd the model wasn't performing the way i thought it would.

[fatbringer]

@Asthestarsfalll i found out that by using paddleseg_utils.load_entire_model(segmodel, segmodel_path) the model now seems to output a more coherent output

May i know what does "load entire model" do exactly?

[Asthestarsfalll]

@fatbringer
It's just a interface to load weights through different argument, such as a URL, or local path, you can use

model.set_state_dict(paddle.load(local_path))

to replace it.

[fatbringer]

@Asthestarsfalll thank you

Another question, i want to try to use the cityscapes SOTA model, the one found at this link https://github.com/PaddlePaddle/PaddleSeg/tree/release/2.8/contrib/CityscapesSOTA

I tried downloading the weights using wget https://bj.bcebos.com/paddleseg/dygraph/cityscapes/mscale_ocr_hrnetw48_cityscapes_autolabel_mapillary/model.pdparams and then using config file mscale_ocr_cityscapes_autolabel_mapillary_ms_val.yml

I got the following error

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
File ~/Desktop/pyenvs/dashboard39/lib/python3.9/site-packages/paddleseg/cvlibs/config_checker.py:39, in ConfigChecker.apply_rule(self, k, cfg)
     38 try:
---> 39     rule.apply(cfg, self.allow_update)
     40 except Exception as e:

File ~/Desktop/pyenvs/dashboard39/lib/python3.9/site-packages/paddleseg/cvlibs/config_checker.py:63, in Rule.apply(self, cfg, allow_update)
     62     cfg = copy.deepcopy(cfg)
---> 63 self.check_and_correct(cfg)

File ~/Desktop/pyenvs/dashboard39/lib/python3.9/site-packages/paddleseg/cvlibs/config_checker.py:73, in DefaultPrimaryRule.check_and_correct(self, cfg)
     72 for i in items:
---> 73     assert i in cfg.dic, \
     74     'No {} specified in the configuration file.'.format(i)

AssertionError: No optimizer specified in the configuration file.

During handling of the above exception, another exception occurred:

RuntimeError                              Traceback (most recent call last)
Cell In[278], line 4
      1 config_file_path = 'paddleSeg_files/mscale_ocr_cityscapes_autolabel_mapillary_ms_val.yml'
      2 segmodel_path = "paddleSeg_files/mscale_ocr_hrnetw48_cityscapes_autolabel_mapillary_model.pdparams"
----> 4 cfg = Config(config_file_path)
      5 builder = SegBuilder(cfg)
      6 segmodel = builder.model

File ~/Desktop/pyenvs/dashboard39/lib/python3.9/site-packages/paddleseg/cvlibs/config.py:83, in Config.__init__(self, path, learning_rate, batch_size, iters, opts, checker)
     81 if checker is None:
     82     checker = self._build_default_checker()
---> 83 checker.apply_all_rules(self)

File ~/Desktop/pyenvs/dashboard39/lib/python3.9/site-packages/paddleseg/cvlibs/config_checker.py:50, in ConfigChecker.apply_all_rules(self, cfg)
     48 def apply_all_rules(self, cfg):
     49     for i in range(len(self.rule_list)):
---> 50         self.apply_rule(i, cfg)

File ~/Desktop/pyenvs/dashboard39/lib/python3.9/site-packages/paddleseg/cvlibs/config_checker.py:41, in ConfigChecker.apply_rule(self, k, cfg)
     39     rule.apply(cfg, self.allow_update)
     40 except Exception as e:
---> 41     raise RuntimeError(
     42         "Sanity check on the configuration file has failed. "
     43         "There should be some problems with your config file. "
     44         "Please check it carefully.\n"
     45         f"The failed rule is {rule.__class__.__name__}, and the error message is: \n{str(e)}"
     46     )

RuntimeError: Sanity check on the configuration file has failed. There should be some problems with your config file. Please check it carefully.
The failed rule is DefaultPrimaryRule, and the error message is: 
No optimizer specified in the configuration file.

I did not change anything in the yaml file. Should i have changed something?

[Asthestarsfalll]

@fatbringer Plz use develop branch

[fatbringer]

@fatbringer Plz use develop branch

@Asthestarsfalll I am sorry I don't understand. What is the 'develop' branch?

[Asthestarsfalll]

@fatbringer Plz use develop branch

@Asthestarsfalll I am sorry I don't understand. What is the 'develop' branch?

The develop branch of this repo, you should install paddleseg via source code

[fatbringer]

Ah ok. But i am trying to use the pretrain weights of the SOTA Cityscapes. Do i still need to do that?

[Asthestarsfalll]

Ah ok. But i am trying to use the pretrain weights of the SOTA Cityscapes. Do i still need to do that?

Cityscapes is a dataset that contain urban street scenes. If you want to segment human, it's better to use pp_humanseg.

And you shouldn't use pretrained weights, the fine-tuned weights is better.

[fatbringer]

@Asthestarsfalll

how should i visualise the items that have TrainID of 255?

   guard rail |  14 |     255 |   construction |          2 |            0 |            1 |   (180, 165, 180)
 
                   bridge |  15 |     255 |   construction |          2 |            0 |            1 |   (150, 100, 100)
 
                   tunnel |  16 |     255 |   construction |          2 |            0 |            1 |    (150, 120, 90)
                    caravan |  29 |     255 |        vehicle |          7 |            1 |            1 |        (0, 0, 90)
 
                  trailer |  30 |     255 |        vehicle |          7 |            1 |            1 |       (0, 0, 110)

I got an out of index error.

[Asthestarsfalll]

@fatbringer
It is impossible to visualise the ignored label in inference.