Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Put load_image function in image_utils.py & fix image rotation issue #14062

Merged
merged 8 commits into from
Nov 3, 2021

Conversation

mishig25
Copy link
Contributor

@mishig25 mishig25 commented Oct 19, 2021

What does this PR do?

  1. Put load_image function in image_utils.py
  2. Encountered and fixing PIL.Image.open is rotating jpeg images PIL.Image.open is rotating jpeg images python-pillow/Pillow#4703

Fixes # (issue)

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines, and
    here are tips on formatting docstrings.
  • Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@mishig25 mishig25 requested review from Narsil and NielsRogge October 19, 2021 12:46
@Narsil
Copy link
Contributor

Narsil commented Oct 19, 2021

Which version of Pillow are you using ? transformers doesn't specify a version.

Pillow 8.0.0 is automatically rotating images upon .open based on EXIF Orientation tag.
Also probably a good time to remove code duplication here.

@mishig25
Copy link
Contributor Author

>>> PIL.__version__
'8.3.1'

also, I get rotated mask results from Inference API, which is how I encountered this issue

@Narsil
Copy link
Contributor

Narsil commented Oct 20, 2021

I think we need to add some form of test to expose the necessity for this as it's not trivial and might be fixed upstream (as it claims to be).

So we need an image file (can be in hf-internal-testing with said exif data) and show that the loading is improper (this test can be entirely separated from the pipeline, here we're testing the load_image method, so we can have a separate class of test for this that focuses on specifically this feature.

@mishig25
Copy link
Contributor Author

mishig25 commented Nov 1, 2021

@Narsil Should I use this opportunity to address (src):

It might be time to create a utils file and put it in there maybe so we can just reuse this code all the time and test it separately. (it should be another PR, just mentioning it here)

If so, should I just make load_image a method of ImageFeatureExtractionMixin & make Vision pipelines inherit ImageFeatureExtractionMixin?

@Narsil
Copy link
Contributor

Narsil commented Nov 1, 2021

Just put the function outside the class, it's a simple function it doesn't deserve a mixin.

In another branch I added src/transformers/audio_utils.py to host ffmpeg_read for instance.

(It's the first thing that popped in mind in terms of location, maybe we can find a better location ultimately, here goal is just to reduce repetition)

@mishig25 mishig25 changed the title Fix img load rotation Put load_image function in image_utils.py & fix image rotation issue Nov 2, 2021
@mishig25
Copy link
Contributor Author

mishig25 commented Nov 2, 2021

@Narsil please feel free to re-review this PR. I've added the changes as suggested.
Specifically, this test case

def test_load_img_exif_transpose(self):
import datasets
dataset = datasets.load_dataset("hf-internal-testing/fixtures_image_utils", "image", split="test")
img_file = dataset[3]["file"]
img_without_exif_transpose = PIL.Image.open(img_file)

Copy link
Contributor

@NielsRogge NielsRogge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! It's indeed better to add the load_image function into image_utils.py.

@NielsRogge NielsRogge requested a review from sgugger November 2, 2021 14:08
Copy link
Collaborator

@sgugger sgugger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to move it there!

tests/test_image_utils.py Outdated Show resolved Hide resolved
tests/test_image_utils.py Outdated Show resolved Hide resolved
Copy link
Member

@LysandreJik LysandreJik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great change, LGTM

@LysandreJik
Copy link
Member

Before merging, there seems to be a few tests failing linked to your PR!

@mishig25
Copy link
Contributor Author

mishig25 commented Nov 3, 2021

@LysandreJik thanks for notifying about the failing tests. Solved! There was an issue with the hf-internal-testing/fixtures_image_utils I've created

@mishig25 mishig25 merged commit 671569d into huggingface:master Nov 3, 2021
enpaul added a commit to enpaul/kodak that referenced this pull request Dec 31, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants