Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Rect object returned by decode can be trivially wrong #51

Open
griai opened this issue Nov 6, 2019 · 15 comments
Open

BUG: Rect object returned by decode can be trivially wrong #51

griai opened this issue Nov 6, 2019 · 15 comments

Comments

@griai
Copy link

griai commented Nov 6, 2019

The rectangle object returned by the decode function is (completely) wrong whenever the found DataMatrix code is rotated. The problem is that the returned rectangle is always supposed to be upright when, in fact, libdtmx supports also rotated codes.

Examples with one upright and one rotated DataMatrix code:

  • direct call of dmtxread on the command line
(base) ➜  ~ dmtxread -n -v -N 1 12345_garbage.png        
--------------------------------------------------
       Matrix Size: 10 x 10
    Data Codewords: 3 (capacity 3)
   Error Codewords: 5
      Data Regions: 1 x 1
Interleaved Blocks: 1
    Rotation Angle: 0
          Corner 0: (1152.0, 1287.0)
          Corner 1: (1201.0, 1287.0)
          Corner 2: (1200.1, 1238.0)
          Corner 3: (1152.0, 1238.0)
--------------------------------------------------
12345

(base) ➜  ~ dmtxread -n -v -N 1 12345_garbage_rotated.png
--------------------------------------------------
       Matrix Size: 10 x 10
    Data Codewords: 3 (capacity 3)
   Error Codewords: 5
      Data Regions: 1 x 1
Interleaved Blocks: 1
    Rotation Angle: 319
          Corner 0: (1191.0, 1292.9)
          Corner 1: (1228.1, 1325.2)
          Corner 2: (1260.5, 1287.9)
          Corner 3: (1223.4, 1255.6)
--------------------------------------------------
12345
  • using pylibdtmx
(libdmtx) ➜  ~ ipython
Python 3.7.5 (default, Oct 25 2019, 15:51:11) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.9.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from pylibdmtx.pylibdmtx import decode                                                                           

In [2]: from PIL import Image                                                                                            

In [3]: decode(Image.open("/home/gri/12345_garbage.png"), max_count=1)                                                   
Out[3]: [Decoded(data=b'12345', rect=Rect(left=1152, top=2220, width=48, height=49))]

In [4]: decode(Image.open("/home/gri/12345_garbage_rotated.png"), max_count=1)                                           
Out[4]: [Decoded(data=b'12345', rect=Rect(left=1191, top=2214, width=70, height=5))]

(Note how also width and height changed to totally incorrect values.)

The obvious solution would be to not return a tuple of left, top, width, and height coordinates, since they apply only to upright rectangles, but to return the coordinates of the four corners---as is done by the command line program.

So in these lines of the code all four points should be captured instead of only two ones.

Of course, such a change would break the interface as it is now.
If you insist on returning an upright rectangle, it should at least be the upright bounding box of the DataMatrix code crop. But I would much prefer if the python version would just return the four corner points as the command line version does.

@EKami
Copy link

EKami commented Jan 15, 2020

Jeez, why they didn't just return the rectangles like in the lib? That would have been so much easier. I think now I'll have to make my own implementation of the lib...

@gauti1311
Copy link

yes you can do by easily adding two more vectors e.g
p00 = DmtxVector2()
p11 = DmtxVector2(1.0, 1.0)
p10 = DmtxVector2(1.0, 0.0)
p01 = DmtxVector2(0.0, 1.0)
dmtxMatrix3VMultiplyBy(p00, region.contents.fit2raw)
dmtxMatrix3VMultiplyBy(p11, region.contents.fit2raw)
dmtxMatrix3VMultiplyBy(p01, region.contents.fit2raw)
dmtxMatrix3VMultiplyBy(p10, region.contents.fit2raw)
x00 = int((shrink * p00.X) + 0.5)
y00 = int((shrink * p00.Y) + 0.5)
x11 = int((shrink * p11.X) + 0.5)
y11 = int((shrink * p11.Y) + 0.5)
x10 = int((shrink * p10.X) + 0.5)
y10 = int((shrink * p10.Y) + 0.5)
x01 = int((shrink * p01.X) + 0.5)
y01 = int((shrink * p01.Y) + 0.5)
return Decoded(
string_at(msg.contents.output),
#Rect2 = namedtuple('Rect', 'x00 y00 x01 y01 x10 y10 x11 y11')
Rect2(x00,y00,x01,y01,x10,y10,x11,y11)

I did its in my lib and it works well

@tomkoenecke
Copy link

tomkoenecke commented Dec 8, 2020

yes you can do by easily adding two more vectors e.g
p00 = DmtxVector2()
p11 = DmtxVector2(1.0, 1.0)
p10 = DmtxVector2(1.0, 0.0)
p01 = DmtxVector2(0.0, 1.0)
dmtxMatrix3VMultiplyBy(p00, region.contents.fit2raw)
dmtxMatrix3VMultiplyBy(p11, region.contents.fit2raw)
dmtxMatrix3VMultiplyBy(p01, region.contents.fit2raw)
dmtxMatrix3VMultiplyBy(p10, region.contents.fit2raw)
x00 = int((shrink * p00.X) + 0.5)
y00 = int((shrink * p00.Y) + 0.5)
x11 = int((shrink * p11.X) + 0.5)
y11 = int((shrink * p11.Y) + 0.5)
x10 = int((shrink * p10.X) + 0.5)
y10 = int((shrink * p10.Y) + 0.5)
x01 = int((shrink * p01.X) + 0.5)
y01 = int((shrink * p01.Y) + 0.5)
return Decoded(
string_at(msg.contents.output),
#Rect2 = namedtuple('Rect', 'x00 y00 x01 y01 x10 y10 x11 y11')
Rect2(x00,y00,x01,y01,x10,y10,x11,y11)

I did its in my lib and it works well

Thanks a ton, this works.
Once i figured out the #Rect2 was meant as the modified definition from further up...

Couldn't this be implemented with an optional activation in the decode function?

@rehman922
Copy link

yes you can do by easily adding two more vectors e.g
p00 = DmtxVector2()
p11 = DmtxVector2(1.0, 1.0)
p10 = DmtxVector2(1.0, 0.0)
p01 = DmtxVector2(0.0, 1.0)
dmtxMatrix3VMultiplyBy(p00, region.contents.fit2raw)
dmtxMatrix3VMultiplyBy(p11, region.contents.fit2raw)
dmtxMatrix3VMultiplyBy(p01, region.contents.fit2raw)
dmtxMatrix3VMultiplyBy(p10, region.contents.fit2raw)
x00 = int((shrink * p00.X) + 0.5)
y00 = int((shrink * p00.Y) + 0.5)
x11 = int((shrink * p11.X) + 0.5)
y11 = int((shrink * p11.Y) + 0.5)
x10 = int((shrink * p10.X) + 0.5)
y10 = int((shrink * p10.Y) + 0.5)
x01 = int((shrink * p01.X) + 0.5)
y01 = int((shrink * p01.Y) + 0.5)
return Decoded(
string_at(msg.contents.output),
#Rect2 = namedtuple('Rect', 'x00 y00 x01 y01 x10 y10 x11 y11')
Rect2(x00,y00,x01,y01,x10,y10,x11,y11)

I did its in my lib and it works well

hi,
beginner in python,can you provide full sample of this implemenation

@jepperaskdk
Copy link

Is there a reason why this bug is still present, 2-3 years after this issue was created?

@gauti1311's solution works for me. @rehman922 you have to alter the pylibdmtx/pylibdmtx.py _decode_region function.
Either return the 4 corners as @gauti1311 or get the boundingbox (preserving the use of Rect) like so:

def _decode_region(decoder, region, corrections, shrink):
    """Decodes and returns the value in a region.

    Args:
        region (DmtxRegion):

    Yields:
        Decoded or None: The decoded value.
    """
    with _decoded_matrix_region(decoder, region, corrections) as msg:
        if msg:
            # Coordinates
            p00 = DmtxVector2()
            p11 = DmtxVector2(1.0, 1.0)
            p10 = DmtxVector2(1.0, 0.0)
            p01 = DmtxVector2(0.0, 1.0)
            dmtxMatrix3VMultiplyBy(p00, region.contents.fit2raw)
            dmtxMatrix3VMultiplyBy(p11, region.contents.fit2raw)
            dmtxMatrix3VMultiplyBy(p01, region.contents.fit2raw)
            dmtxMatrix3VMultiplyBy(p10, region.contents.fit2raw)
            x00 = int((shrink * p00.X) + 0.5)
            y00 = int((shrink * p00.Y) + 0.5)
            x11 = int((shrink * p11.X) + 0.5)
            y11 = int((shrink * p11.Y) + 0.5)
            x10 = int((shrink * p10.X) + 0.5)
            y10 = int((shrink * p10.Y) + 0.5)
            x01 = int((shrink * p01.X) + 0.5)
            y01 = int((shrink * p01.Y) + 0.5)

            min_x = min(x00, x11, x10, x01)
            max_x = max(x00, x11, x10, x01)
            min_y = min(y00, y11, y10, y01)
            max_y = max(y00, y11, y10, y01)

            return Decoded(
                string_at(msg.contents.output),
                Rect(min_x, min_y, max_x - min_x, min_y - max_y)
            )
        else:
            return None

@fadamsyah
Copy link

Is there a reason why this bug is still present, 2-3 years after this issue was created?

@gauti1311's solution works for me. @rehman922 you have to alter the pylibdmtx/pylibdmtx.py _decode_region function. Either return the 4 corners as @gauti1311 or get the boundingbox (preserving the use of Rect) like so:

def _decode_region(decoder, region, corrections, shrink):
    """Decodes and returns the value in a region.

    Args:
        region (DmtxRegion):

    Yields:
        Decoded or None: The decoded value.
    """
    with _decoded_matrix_region(decoder, region, corrections) as msg:
        if msg:
            # Coordinates
            p00 = DmtxVector2()
            p11 = DmtxVector2(1.0, 1.0)
            p10 = DmtxVector2(1.0, 0.0)
            p01 = DmtxVector2(0.0, 1.0)
            dmtxMatrix3VMultiplyBy(p00, region.contents.fit2raw)
            dmtxMatrix3VMultiplyBy(p11, region.contents.fit2raw)
            dmtxMatrix3VMultiplyBy(p01, region.contents.fit2raw)
            dmtxMatrix3VMultiplyBy(p10, region.contents.fit2raw)
            x00 = int((shrink * p00.X) + 0.5)
            y00 = int((shrink * p00.Y) + 0.5)
            x11 = int((shrink * p11.X) + 0.5)
            y11 = int((shrink * p11.Y) + 0.5)
            x10 = int((shrink * p10.X) + 0.5)
            y10 = int((shrink * p10.Y) + 0.5)
            x01 = int((shrink * p01.X) + 0.5)
            y01 = int((shrink * p01.Y) + 0.5)

            min_x = min(x00, x11, x10, x01)
            max_x = max(x00, x11, x10, x01)
            min_y = min(y00, y11, y10, y01)
            max_y = max(y00, y11, y10, y01)

            return Decoded(
                string_at(msg.contents.output),
                Rect(min_x, min_y, max_x - min_x, min_y - max_y)
            )
        else:
            return None

Maybe because there's no pull request on this issue(?)

@Gennadynemchin
Copy link

Hi Everyone! I tried all of the methods above but anyway have wrong coordinates. Could anyone suggest a solution?

изображение

@Gennadynemchin
Copy link

Solved!

@GenericArt
Copy link

Solved!

How did you solve it?

@amackp
Copy link

amackp commented Sep 29, 2022

Solved!

How did you solve it?

@GenericArt Not sure what @Gennadynemchin 's actual issue was, but I found that the when using @gauti1311 's solution to return 4 corners from the _decode_region, the y coordinate frame is flipped relative to the coordinate frame used by openCV images. You can simply subtract the input image's height from the rectangle's Y coordinates to get the proper pixel location.

cv_image = cv2.imread('/path/to/image.png')

results = pylibdmtx.decode(cv_image)

height, width, channels = cv_image.shape

for code in results:
    tl = (code.rect2.x01, height - code.rect2.y01)
    tr = (code.rect2.x11, height - code.rect2.y11)
    br = (code.rect2.x10, height - code.rect2.y10)
    bl = (code.rect2.x00, height - code.rect2.y00)
   
    # do something with rectangle... 

@GenericArt
Copy link

Solved!

How did you solve it?

@GenericArt Not sure what @Gennadynemchin 's actual issue was, but I found that the when using @gauti1311 's solution to return 4 corners from the _decode_region, the y coordinate frame is flipped relative to the coordinate frame used by openCV images. You can simply subtract the input image's height from the rectangle's Y coordinates to get the proper pixel location.

cv_image = cv2.imread('/path/to/image.png')

results = pylibdmtx.decode(cv_image)

height, width, channels = cv_image.shape

for code in results:
    tl = (code.rect2.x01, height - code.rect2.y01)
    tr = (code.rect2.x11, height - code.rect2.y11)
    br = (code.rect2.x10, height - code.rect2.y10)
    bl = (code.rect2.x00, height - code.rect2.y00)
   
    # do something with rectangle... 

Ah, I thought an update to the code had been implemented. Nevermind.

@Gennadynemchin
Copy link

You are right - I flipped the image. That's the only solution I've found

`while camera.IsGrabbing():
grabResult = camera.RetrieveResult(5000, pylon.TimeoutHandling_ThrowException)
t0 = time.time()
if grabResult.GrabSucceeded():
image = converter.Convert(grabResult)
img = image.GetArray()
img_flip = cv2.flip(img, 0)
codes = decode(img, timeout=8000, max_count=99, corrections=3)

do something

cv2.imshow('dm', img_flip)`

@jepperaskdk
Copy link

@Gennadynemchin Usually this happens if two libraries define the origin differently, e.g. OpenCV sets it in top-left and perhaps this library returns coordinates with origin in bottom-left. You then need to invert the Y-axis (subtracting the coordinate from the height of the image) to get the same coordinates.

@jepperaskdk
Copy link

@pcumani has opened a pull-request to solve this issue #85. Could you all help get it merged in? I would prefer not having to roll with a modified version of this library.

@JonZavialov
Copy link

Few months late to this but I recently needed this functionality, strange how they haven't merged it in yet. I didn't want to modify the actual library so here is my implementation, thanks to @jepperaskdk and @amackp:

from pylibdmtx.pylibdmtx import _region, _decoder, _image, _pixel_data, _decoded_matrix_region
from pylibdmtx.wrapper import c_ubyte_p, DmtxPackOrder, DmtxVector2, dmtxMatrix3VMultiplyBy, DmtxUndefined
from ctypes import cast, string_at
from collections import namedtuple
import numpy

_pack_order = {
    8: DmtxPackOrder.DmtxPack8bppK,
    16: DmtxPackOrder.DmtxPack16bppRGB,
    24: DmtxPackOrder.DmtxPack24bppRGB,
    32: DmtxPackOrder.DmtxPack32bppRGBX,
}
Decoded = namedtuple('Decoded', 'data rect')


def decode_with_region(image):
    results = []
    pixels, width, height, bpp = _pixel_data(image)
    with _image(cast(pixels, c_ubyte_p), width, height, _pack_order[bpp]) as img:
        with _decoder(img, 1) as decoder:
            while True:
                with _region(decoder, None) as region:
                    if not region:
                        break
                    else:
                        res = _decode_region(decoder, region)
                        if res:
                            open_cv_image = numpy.array(image)
                            # Convert RGB to BGR
                            open_cv_image = open_cv_image[:, :, ::-1].copy()
                            height, width, _ = open_cv_image.shape

                            topLeft = (res.rect['01']['x'], height - res.rect['01']['y'])
                            topRight = (res.rect['11']['x'], height - res.rect['11']['y'])
                            bottomRight = (res.rect['10']['x'], height - res.rect['10']['y'])
                            bottomLeft = (res.rect['00']['x'], height - res.rect['00']['y'])
                            results.append(Decoded(res.data, (topLeft, topRight, bottomRight, bottomLeft)))
    return results


def _decode_region(decoder, region):
    with _decoded_matrix_region(decoder, region, DmtxUndefined) as msg:
        if msg:
            vector00 = DmtxVector2()
            vector11 = DmtxVector2(1.0, 1.0)
            vector10 = DmtxVector2(1.0, 0.0)
            vector01 = DmtxVector2(0.0, 1.0)
            dmtxMatrix3VMultiplyBy(vector00, region.contents.fit2raw)
            dmtxMatrix3VMultiplyBy(vector11, region.contents.fit2raw)
            dmtxMatrix3VMultiplyBy(vector01, region.contents.fit2raw)
            dmtxMatrix3VMultiplyBy(vector10, region.contents.fit2raw)

            return Decoded(
                string_at(msg.contents.output),
                {
                    '00': {
                        'x': int((vector00.X) + 0.5),
                        'y': int((vector00.Y) + 0.5)
                    },
                    '01': {
                        'x': int((vector01.X) + 0.5),
                        'y': int((vector01.Y) + 0.5)
                    },
                    '10': {
                        'x': int((vector10.X) + 0.5),
                        'y': int((vector10.Y) + 0.5)
                    },
                    '11': {
                        'x': int((vector11.X) + 0.5),
                        'y': int((vector11.Y) + 0.5)
                    }
                }
            )
        else:
            return None

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests