-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multimodal model support #249
Conversation
@@ -38,10 +38,13 @@ def test_simple_index_creation(self): | |||
expected_settings = { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you have at least one test on the error message?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
"type": "structured", | ||
"model": "LanguageBind/Video_V1.5_FT_Audio_FT_Image", | ||
"inferenceType": "marqo.GPU", | ||
"storageClass": "marqo.performance", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are we using performance?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thought it would be good for test coverage. I just switched to balanced
What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
introduces new field for create index:
treat_urls_and_pointers_as_media
(treat_urls_and_pointers_as_images
is still supported). IntroducesaudioPreprocessing
andvideoPreprocessing
forstructured
andunstructured
indexes.What is the current behavior? (You can also link to an open issue here)
We only have
treat_urls_and_pointers_as_images
and no index settings for audio and video preprocessing.What is the new behavior (if this is a feature change)?
In order to get marqo to download and process video and audio files from urls, we need
treat_urls_and_pointers_as_media
. In fulfillment of marqo 2.12 PR Add multimodal model support marqo#931In order to allow the user to customize preprocessing fro audio and video files.
Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)
TBD
Other information: