Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Audio: OpenAI API #377

Merged
merged 12 commits into from
Sep 24, 2024
Merged

Audio: OpenAI API #377

merged 12 commits into from
Sep 24, 2024

Conversation

michaelfeil
Copy link
Owner

@michaelfeil michaelfeil commented Sep 24, 2024

OpenAI does not have image embeddings, therefore using an API similar to embeddings.

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

This PR adds audio embedding support to the Infinity API, updates the OpenAPI specification, and introduces asynchronous operations for image and audio processing to improve performance.

  • Added '/embeddings_audio' endpoint in infinity_server.py for audio embedding, but implementation has issues needing attention
  • Replaced 'requests' with 'aiohttp' in _optional_imports.py for asynchronous HTTP requests
  • Introduced AudioEmbeddingInput class in pymodels.py, inheriting from ImageEmbeddingInput
  • Updated resolve_images and resolve_audios functions in utils.py to use asynchronous operations with aiohttp and asyncio.gather()
  • Modified BatchHandler in batch_handler.py to directly await resolve_images and resolve_audios, removing asyncio.to_thread

6 file(s) reviewed, 5 comment(s)
Edit PR Review Bot Settings

libs/infinity_emb/infinity_emb/infinity_server.py Outdated Show resolved Hide resolved
libs/infinity_emb/infinity_emb/infinity_server.py Outdated Show resolved Hide resolved
libs/infinity_emb/infinity_emb/infinity_server.py Outdated Show resolved Hide resolved
libs/infinity_emb/infinity_emb/infinity_server.py Outdated Show resolved Hide resolved
@codecov-commenter
Copy link

codecov-commenter commented Sep 24, 2024

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 21.73913% with 18 lines in your changes missing coverage. Please review.

Project coverage is 77.78%. Comparing base (8690ebc) to head (b3ea3a2).
Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
libs/infinity_emb/infinity_emb/infinity_server.py 10.52% 17 Missing ⚠️
...inity_emb/infinity_emb/fastapi_schemas/pymodels.py 75.00% 1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #377      +/-   ##
==========================================
- Coverage   78.24%   77.78%   -0.47%     
==========================================
  Files          37       37              
  Lines        2781     2804      +23     
==========================================
+ Hits         2176     2181       +5     
- Misses        605      623      +18     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@michaelfeil michaelfeil merged commit 978757d into main Sep 24, 2024
@michaelfeil michaelfeil deleted the mf-async-requests branch September 24, 2024 05:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants