Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support loading base64 images in pipelines #25633

Merged
merged 22 commits into from
Aug 29, 2023

Conversation

InventivetalentDev
Copy link
Contributor

What does this PR do?

Adds support for loading base64-encoded images in pipelines.
I primarily added this so it can be used by transformers-cli serve, since I found having to use an URL or reference to a local file a bit inconvenient for my use-case.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines, and
    here are tips on formatting docstrings.
  • Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@Narsil

@ArthurZucker
Copy link
Collaborator

cc @amyeroberts

Copy link
Collaborator

@amyeroberts amyeroberts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this!

Really useful functionality - main comment is to update all logic of the function e.g. errors raised and make it a bit more defensive.

tests/utils/test_image_utils.py Outdated Show resolved Hide resolved
src/transformers/image_utils.py Show resolved Hide resolved
src/transformers/image_utils.py Outdated Show resolved Hide resolved
tests/fixtures/base64_image.txt Outdated Show resolved Hide resolved
@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

Copy link
Collaborator

@amyeroberts amyeroberts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this!

cc @Narsil for a second opinion on the changes for the pipeline

@Narsil
Copy link
Contributor

Narsil commented Aug 23, 2023

The PR looks good.

I'm uneasy about the actual format suggested, because some stuff are base64 decodable and yet real strings.
I feel like this should be in user code not really in transformers.

But I have the same feeling about loading images from URL so..

@amyeroberts
Copy link
Collaborator

@Narsil Understood! As passing in strings is already supported for loading from url, I think this is an acceptable addition. Is there anything you'd like us to add to this PR to handle that?

@Narsil
Copy link
Contributor

Narsil commented Aug 24, 2023

No not really.

It's choice. I'm not in favor, but I'm not super strongly opposed to it either.
Please choose which route you prefer.

@amyeroberts
Copy link
Collaborator

@Narsil I'm inclined to agree. As we already accept strings, I'm going to merge. If continue support of this ends up being a headache (lots of code / lots of if/else logic) then we can think about ways to deprecate the support or reduce its scope.

@amyeroberts amyeroberts merged commit dbc16f4 into huggingface:main Aug 29, 2023
21 checks passed
@InventivetalentDev InventivetalentDev deleted the serving-image-base64 branch August 29, 2023 22:16
parambharat pushed a commit to parambharat/transformers that referenced this pull request Sep 26, 2023
* support loading base64 images

* add test

* mention in docs

* remove the logging

* sort imports

* update error message

* Update tests/utils/test_image_utils.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* restructure to catch base64 exception

* doesn't like the newline

* download files

* format

* optimize imports

* guess it needs a space?

* support loading base64 images

* add test

* remove the logging

* sort imports

* restructure to catch base64 exception

* doesn't like the newline

* download files

* optimize imports

* guess it needs a space?

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
blbadger pushed a commit to blbadger/transformers that referenced this pull request Nov 8, 2023
* support loading base64 images

* add test

* mention in docs

* remove the logging

* sort imports

* update error message

* Update tests/utils/test_image_utils.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* restructure to catch base64 exception

* doesn't like the newline

* download files

* format

* optimize imports

* guess it needs a space?

* support loading base64 images

* add test

* remove the logging

* sort imports

* restructure to catch base64 exception

* doesn't like the newline

* download files

* optimize imports

* guess it needs a space?

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
EduardoPach pushed a commit to EduardoPach/transformers that referenced this pull request Nov 18, 2023
* support loading base64 images

* add test

* mention in docs

* remove the logging

* sort imports

* update error message

* Update tests/utils/test_image_utils.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* restructure to catch base64 exception

* doesn't like the newline

* download files

* format

* optimize imports

* guess it needs a space?

* support loading base64 images

* add test

* remove the logging

* sort imports

* restructure to catch base64 exception

* doesn't like the newline

* download files

* optimize imports

* guess it needs a space?

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants