update the SemanticSegmentationTranslator #2032
Conversation
Codecov ReportBase: 72.08% // Head: 70.24% // Decreases project coverage by
Additional details and impacted files@@ Coverage Diff @@
## master #2032 +/- ##
============================================
- Coverage 72.08% 70.24% -1.85%
- Complexity 5126 6075 +949
============================================
Files 473 603 +130
Lines 21970 26892 +4922
Branches 2351 2912 +561
============================================
+ Hits 15838 18890 +3052
- Misses 4925 6591 +1666
- Partials 1207 1411 +204
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report at Codecov. |
xyang16
force-pushed
the
update_semanticSegmentationTranslator
branch
3 times, most recently
from
f4030fb
to
6c559ef
Compare
| .registerTypeAdapter(Segmentation.class, new SegmentationSerializer()) | ||
| .create(); | ||
|
|
||
| private int[][] mask; |
There was a problem hiding this comment.
Segmentation should hold classNames
|
|
||
| private final int shortEdge; | ||
| private final int maxEdge; | ||
|
|
||
| private static final int CHANNEL = 3; | ||
| private static final int CLASSNUM = 21; |
There was a problem hiding this comment.
This should comes from model, for deeplabv3, the classes can be found: https://github.com/pytorch/vision/blob/main/torchvision/models/_meta.py#L1128-L1150
| double maxnum = -Double.MAX_VALUE; | ||
| for (int i = 0; i < CLASSNUM; i++) { | ||
| // get score for each i at the h,w pixel of the image | ||
| float score = scores[i * (imageSize) + index]; |
There was a problem hiding this comment.
| float score = scores[i * (imageSize) + index]; | |
| float score = scores[i * imageSize + index]; |
| public Segmentation(int[][] mask) { | ||
| this.mask = mask; | ||
| } | ||
|
|
There was a problem hiding this comment.
We need implement the following method:
public void drawMask(Image img, float transparency) {
...
}
public void drawMask(Image img, float transparency, int classId) {
...
}
public void drawMask(Image img, float transparency, int classId, Color color) {
...
}
public void Image extractImage(Image img, int classId) {
...
}
| } | ||
| } | ||
|
|
||
| private static void saveSemanticImage(Image img) throws IOException { |
There was a problem hiding this comment.
We still want to save segmented image, and we can save a few different mode:
- only colors
- transparent color overlay on existing image
- replace background with other image
xyang16
force-pushed
the
update_semanticSegmentationTranslator
branch
from
8c60b7f
to
4c16f9b
Compare
| @@ -120,6 +120,12 @@ public Image fromNDArray(NDArray array) { | |||
| return new BitMapWrapper(bitmap); | |||
| } | |||
|
|
|||
| /** {@inheritDoc} */ | |||
| @Override | |||
| public Image fromArray(int[] pixels, int width, int height) { | |||
There was a problem hiding this comment.
Android depends on published snapshot build.
Move android changes into separate PR.
| @@ -157,6 +165,16 @@ public Object getWrappedImage() { | |||
| return image; | |||
| } | |||
|
|
|||
| @Override | |||
There was a problem hiding this comment.
| @Override | |
| /** {@inheritDoc} */ | |
| @Override |
xyang16
force-pushed
the
update_semanticSegmentationTranslator
branch
6 times, most recently
from
cdc0d4c
to
97791e7
Compare
frankfliu
force-pushed
the
update_semanticSegmentationTranslator
branch
from
0e7470f
to
3501500
Compare
Description
Brief description of what this PR is about
update the SemanticSegmentationTranslator to get the original photo pixels