Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add intra-run multithreading to SearchCollection #1678

Merged
merged 9 commits into from
Dec 2, 2021
Merged

Conversation

lintool
Copy link
Member

@lintool lintool commented Nov 19, 2021

Context: SearchCollection currently has the ability to do parameter sweeps, with different configurations in parallel using a thread pool. However, it currently has no ability to run different queries in a single configuration in parallel. This means that some sparse models take a long time to run.

This PR adds a threadpool within an individual configuration to run queries in parallel.

@lintool lintool requested a review from MXueguang November 19, 2021 14:45
@lintool lintool marked this pull request as draft November 19, 2021 14:45
@codecov-commenter
Copy link

codecov-commenter commented Nov 19, 2021

Codecov Report

Merging #1678 (5bad21d) into master (f7970fb) will decrease coverage by 0.13%.
The diff coverage is 51.54%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #1678      +/-   ##
============================================
- Coverage     57.36%   57.23%   -0.14%     
  Complexity     1029     1029              
============================================
  Files           172      172              
  Lines          9618     9668      +50     
  Branches       1324     1336      +12     
============================================
+ Hits           5517     5533      +16     
- Misses         3637     3666      +29     
- Partials        464      469       +5     
Impacted Files Coverage Δ
...rc/main/java/io/anserini/search/SearchMsmarco.java 0.00% <0.00%> (ø)
...main/java/io/anserini/search/SearchCollection.java 43.92% <51.64%> (-1.28%) ⬇️
...c/main/java/io/anserini/index/IndexCollection.java 45.01% <100.00%> (ø)
src/main/java/io/anserini/search/SearchArgs.java 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f7970fb...5bad21d. Read the comment docs.

@lintool lintool requested review from Peilin-Yang and removed request for MXueguang November 30, 2021 23:07
@lintool lintool marked this pull request as ready for review November 30, 2021 23:08
@lintool lintool requested a review from MXueguang November 30, 2021 23:08
@lintool
Copy link
Member Author

lintool commented Nov 30, 2021

All regressions successfully passed with the multi-threaded implementation.

@lintool lintool merged commit e33f1ff into master Dec 2, 2021
@lintool lintool deleted the multithreaded branch December 2, 2021 03:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants