Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/clip #246

Merged
merged 20 commits into from
Jan 21, 2022
Merged

Feature/clip #246

merged 20 commits into from
Jan 21, 2022

Conversation

lucaro
Copy link
Member

@lucaro lucaro commented Jan 13, 2022

  • Added Image and Text features based on OpenAI CLIP
  • refactored and reorganized some common helper logic

silvanheller and others added 18 commits January 4, 2022 09:13
@silvanheller
bumping version number
7805260
@lucaro @silvanheller
Added ThreadLocalObjectCache to reuse objects in a thread-safe manner
d10d914
@Spiess @silvanheller
Implemented visual-text co-embedding extraction of full video segment…
201edae 
…s rather than only the most representative frame.
@Spiess @silvanheller
Implemented early return to conform to codebase guidelines.
1ecf39a
@Spiess @silvanheller
Refactored visual-text co-embedding to load buffered image only once …
f774eb9 
…needed.
@Spiess @silvanheller
Removed unused and empty JavaDoc.
e442ec5
@Spiess @silvanheller
Removed unused commented code.
8a2ff57
@Spiess @silvanheller
Fixed JavaDoc errors.
0d9e982
@Spiess @silvanheller
Raised source and target language compatibility to Java 9.
6a2c5cd 
This is required since FFMpegAudioDecoder and TechnicalVideoMetadataExtractor use Math.floorDiv, which is a function only available since Java 9.
@Spiess @silvanheller
Raised source and target language compatibility to Java 11 (LTS).
4b2e801 
Due to popular demand.
@Spiess @silvanheller
Reformatted all code to adhere to the project code-style.
ef8d793
@Spiess
Fixed CottontailWrapper pretending connection was successful even whe…
cea3083 
…n it wasn't.
@lucaro
Added feature modules for OpenAI CLIP
a14ab03
@lucaro
Added CLIP models to external files
b3af674
@lucaro
Minor refactoring of utility classes
ef0539f
@lucaro
Merge master into dev
baa963e 
# Conflicts:
#	build.gradle
#	cineast-core/src/main/java/org/vitrivr/cineast/core/features/VisualTextCoEmbedding.java
@lucaro
Merge branch 'dev' into feature/clip
c3897af
@lucaro
Refactored common image preprocessing logic into helper class
3132b2d
@lucaro lucaro requested review from Spiess and silvanheller January 13, 2022 16:02
Spiess
Spiess reviewed Jan 20, 2022
View reviewed changes
Copy link
Member

@Spiess Spiess left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thank you very much for the work.

I added two comments about System.outs that may have been left in from debugging. Once they are either removed or I have your confirmation that they exist intentionally I will be happy to give my approval.

cineast-core/src/main/java/org/vitrivr/cineast/core/features/CLIPImage.java Outdated Show resolved Hide resolved
cineast-core/src/main/java/org/vitrivr/cineast/core/features/CLIPText.java Outdated Show resolved Hide resolved
@lucaro lucaro requested a review from Spiess January 20, 2022 14:53
Spiess
Spiess previously approved these changes Jan 20, 2022
View reviewed changes
Copy link
Member

@Spiess Spiess left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks again!

@silvanheller
Copy link
Member

Could this target master directly? I don't see a reason this has to target dev first

@lucaro
Copy link
Member Author

lucaro commented Jan 21, 2022

Could this target master directly? I don't see a reason this has to target dev first

I guess it could, I don't really have a preference for the target. If we want to make a 'minor release' out of this, sure.

@silvanheller silvanheller changed the base branch from dev to master January 21, 2022 15:28
@silvanheller silvanheller dismissed Spiess’s stale review January 21, 2022 15:28

The base branch was changed.

silvanheller
silvanheller approved these changes Jan 21, 2022
View reviewed changes
Copy link
Member

@silvanheller silvanheller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the refactoring work!

@silvanheller silvanheller merged commit c2315f0 into master Jan 21, 2022
@silvanheller silvanheller deleted the feature/clip branch January 21, 2022 15:49
silvanheller added a commit that referenced this pull request Jun 10, 2022
@lucaro @silvanheller @Spiess
CLIP feature module (#246)
b504855 
* Added feature modules for OpenAI CLIP

* Minor refactoring of utility classes

* Refactored common image preprocessing logic into helper class

Co-authored-by: Silvan Heller <silvan.heller@unibas.ch>
Co-authored-by: Florian Spiess <florian.spiess@unibas.ch>
Former-commit-id: c2315f0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@silvanheller silvanheller silvanheller approved these changes

@Spiess Spiess Spiess approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@lucaro @silvanheller @Spiess