Allow access of signed 16-bit TIFF pixel data as NumPy arrays

[tomgoddard]

Small additions to access I;16S TIFF pixel data as numpy arrays.
Code to convert to other pixel types such as 32-bit integers is not included.
This pull request doesn't fix all the deficiencies in handling 16-bit signed TIFF data but it is a start. I use this in a patched Pillow version in a visualization package ChimeraX for light microscopy data.

Partial fix to issue #861

Related to closed issue #273 addressed by merged pull request #347

Changes proposed in this pull request:

• This enables opening a TIFF image containing mode I;16S pixels and seeking in multipage TIFF:

  i = Image.open('test_i16s.tif')
  i.seek(1)
  import numpy
  a = numpy.array(i)

The current Pillow master branch and 4.1.1 release gives an error on the seek and a zero dimensional array when trying to get the pixels. This pull request gives the correct results. Here is a multipage I;16S tiff example file.

http://www.cgl.ucsf.edu/home/goddard/temp/test_i16s.tif

Here is the error using from the seek() call using Pillow 4.1.1

>>> i = Image.open('test_i16s.tif')
>>> i.seek(1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/site-packages/PIL/TiffImagePlugin.py", line 950, in seek
    self.im = Image.core.new(self.mode, self.size)
ValueError: unrecognized mode
>>> i.mode
'I;16S'

and the incorrect result of the numpy array call in Pillow 4.1.1

>>> a = numpy.array(i)
>>> a
array(<PIL.TiffImagePlugin.TiffImageFile image mode=I;16S size=300x512 at 0x109D3D588>, dtype=object)
>>> a.shape
()

[hugovk]

@tomgoddard Please can you add unit tests?

[wiredfool]

This is discussed in #2228

[tomgoddard]

I added conversion code between I;16S and modes I and L and added I;16S to the test_mode_i16.py unit test.

[tomgoddard]

I added tests of the I;16S mode a week ago. Is there more I need to do to move this along?

[homm]

In general I against adding modes with im->type = IMAGING_TYPE_SPECIAL. I'd prefer to have less fully supported modes than more modes with 1-2 applicable operations. In my opinion Pillow must support all possible external modes and convert them to one of the appropriate internal modes. Also Pillow should allow to export images as bytes or numpy arrays in wide range of modes.

For example, in case of I;16S TIFFs, Pillow should load them in I mode (which is already signed, I believe) and provide some way to construct numpy array in I;16S format. As I see there is a problem with exporting numpy arrays in arbitrary mode, because Image.__array_interface__ acts based on current mode and there in no way to specify another. Additionally, I;16S mode is one of the I;16 family, which for some reasons, was not added on Storage.c and Convert.c.

I think we should accept this.

[homm]

Please check, that you are using 4 spaces as indention everywhere.

[homm]

wrong indention

[homm]

wrong indention

[wiredfool]

I've been looking at this and adding more tests, access, and whatnot.

I'm now leaning against this approach.

Prior to this PR, I think the core issue is in the tiff OPEN_INFO specification, specifically this:

    (II, 1, (1,), 1, (16,), ()): ("I;16", "I;16"),
    (MM, 1, (1,), 1, (16,), ()): ("I;16B", "I;16B"),
    (II, 1, (2,), 1, (16,), ()): ("I;16S", "I;16S"),
    (MM, 1, (2,), 1, (16,), ()): ("I;16BS", "I;16BS"),

The last to elements are (mode, rawmode). These modes have been here since The Fork, but we've never had a mode for either of the S modes. We have had rawmodes in Unpack.c for these formats to unpack to an I mode the whole time.

Also, mirroring this is the Image._fromarray_typemap, which maps all of the I;16 modes to I mode:

    ((1, 1), "<u2"): ("I", "I;16"),
    ((1, 1), ">u2"): ("I", "I;16B"),
    ((1, 1), "<i2"): ("I", "I;16S"),
    ((1, 1), ">i2"): ("I", "I;16BS"),

Though, we do optimistically offer a bunch of rawmodes in the Image._MODE_CONV map to transfer a whole host of rawmodes to numpy. Unfortunately, this is only ever accessed with self.mode, which is not a rawmode.

    "I;16S": ('<i2', None),
    "I;16BS": ('>i2', None),
    "I;16LS": ('<i2', None),
    "I;32": ('<u4', None),
    "I;32B": ('>u4', None),
    "I;32L": ('<u4', None),
    "I;32S": ('<i4', None),
    "I;32BS": ('>i4', None),
    "I;32LS": ('<i4', None),

Also, there's this one other mode, which is broken as well:

    (MM, 1, (2,), 1, (32,), ()): ("I;32BS", "I;32BS"),

There's got to be a better solution to this than adding a supported mode which doesn't work with any of the Image operations that we do.

Option 1 is just to fix the TiffImagePlugin to not advertise modes we don't support. We could losslessly read an I;16S into an I image, do any image manipulations on it, and export it to numpy as an int32. If they wish to change the type in numpy, that's an easy operation.

Alternately, we could do something along the lines of an adapter that provides an __array_interface__ of a specified rawmode from an Image. Where now, the interface looks like np.array(img), it would be something like: np.array(img.as_array_mode('I;16S')). This would essentially do the image type conversion in Pillow. rather than in numpy.

I'm hesitant to add a lightly supported mode like I;16S that has such a potential for numerical error.

[tomgoddard]

For my uses being able to access I;16S TIFF files as int32 arrays would be fine. Does the comment "We have had rawmodes in Unpack.c for these formats to unpack to an I mode the whole time.” mean there is some way to read I;16S data currently? The following code using Image.convert(‘I’) on an I;16S TIFF image gives an error in Pillow 4.2.1 path = '/Users/goddard/Downloads/test_i16s.tif' from PIL import Image i = Image.open(path).convert(‘I') File "test.py", line 3, in <module> i = Image.open(path).convert('I') File "/Users/goddard/ucsf/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/PIL/Image.py", line 860, in convert self.load() File "/Users/goddard/ucsf/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/PIL/TiffImagePlugin.py", line 1018, in load return super(TiffImageFile, self).load() File "/Users/goddard/ucsf/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/PIL/ImageFile.py", line 188, in load self.load_prepare() File "/Users/goddard/ucsf/chimerax/ChimeraX.app/Contents/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/PIL/ImageFile.py", line 261, in load_prepare self.im = Image.core.new(self.mode, self.size) ValueError: unrecognized mode

[wiredfool]

These lines in TiffImagePlugin should be changed to make it work:

-    (II, 1, (2,), 1, (16,), ()): ("I;16S", "I;16S"),
-    (MM, 1, (2,), 1, (16,), ()): ("I;16BS", "I;16BS"),
+    (II, 1, (2,), 1, (16,), ()): ("I", "I;16S"),
+    (MM, 1, (2,), 1, (16,), ()): ("I", "I;16BS"),

[wiredfool]

(I'll note that it's likely completely untested, but I believe that there might be a way to trigger that code in currently shipping Pillow by doing raw conversions from a buffer or bytes.)

[tomgoddard]

Thanks! This two line change solves the problem in my software reading 16-bit signed TIFF microscopy images. With this change numpy.array(image) produces 32-bit int arrays. It would be better if 16-bit int arrays were directly produced but 32-bit int is a perfectly workable solution. The advantage of 16-bit is that these can be giga-byte size multiimage stacks (3-dimensional microscopy) so efficiently reading into a native 16-bit int array is preferred for performance reasons.

[wiredfool]

I'm closing this, there's a successor PR (#2743) for it including some other testing, but the other approach was merged (#2748)