-
Notifications
You must be signed in to change notification settings - Fork 123
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Embed openai broad multimodal compat #395
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
This pull request introduces significant changes to support multimodal embeddings (text, image, and audio) through a unified /embeddings
endpoint in the Infinity API.
- Deprecated separate
/embeddings_image
and/embeddings_audio
endpoints in favor of the unified/embeddings
endpoint withinfinity_extra_modality
parameter - Added new
OpenAIEmbeddingInputAudio
,OpenAIEmbeddingInputImage
, andOpenAIEmbeddingInputText
models inlibs/client_infinity/infinity_client/infinity_client/models/
- Updated
infinity_server.py
to handle multimodal inputs and improve error handling - Introduced comprehensive tests in
test_openapi_client_compat.py
for OpenAI-compatible embedding API across all modalities - Updated OpenAPI specification in
docs/assets/openapi.json
to reflect new multimodal embedding capabilities
27 file(s) reviewed, 8 comment(s)
Edit PR Review Bot Settings
libs/client_infinity/infinity_client/infinity_client/models/open_ai_embedding_input_audio.py
Show resolved
Hide resolved
...inity_client/infinity_client/models/open_ai_embedding_input_audio_infinity_extra_modality.py
Show resolved
Hide resolved
libs/client_infinity/infinity_client/infinity_client/models/open_ai_embedding_input_image.py
Show resolved
Hide resolved
...inity_client/infinity_client/models/open_ai_embedding_input_image_infinity_extra_modality.py
Show resolved
Hide resolved
...finity_client/infinity_client/models/open_ai_embedding_input_text_infinity_extra_modality.py
Show resolved
Hide resolved
Codecov ReportAttention: Patch coverage is
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## main #395 +/- ##
==========================================
- Coverage 78.67% 78.55% -0.12%
==========================================
Files 39 39
Lines 2963 3008 +45
==========================================
+ Hits 2331 2363 +32
- Misses 632 645 +13 ☔ View full report in Codecov by Sentry. |
ea35e76
to
a23810b
Compare
Closes #394