Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix race in AbstractSearchAsyncAction request throttling #116264

Merged
merged 3 commits into from
Nov 7, 2024
Merged

Fix race in AbstractSearchAsyncAction request throttling #116264

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ce4b14e
Fix race in AbstractSearchAsyncAction request throttling
original-brownbear Nov 5, 2024
9db4a9e
Merge remote-tracking branch 'elastic/main' into fix-race-abstract-se…
original-brownbear Nov 7, 2024
523167d
cleanup
original-brownbear Nov 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions muted-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -237,9 +237,6 @@ tests:
- class: org.elasticsearch.search.basic.SearchWithRandomDisconnectsIT
method: testSearchWithRandomDisconnects
issue: https://github.com/elastic/elasticsearch/issues/116175
- class: org.elasticsearch.indexing.IndexActionIT
method: testAutoGenerateIdNoDuplicates
issue: https://github.com/elastic/elasticsearch/issues/115716
- class: org.elasticsearch.xpack.test.rest.XPackRestIT
method: test {p0=ml/start_stop_datafeed/Test start datafeed given index pattern with no matching indices}
issue: https://github.com/elastic/elasticsearch/issues/116220
Expand Down
17 changes: 12 additions & 5 deletions server/src/main/java/org/elasticsearch/action/search/AbstractSearchAsyncAction.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,8 +48,8 @@
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.Executor;
import java.util.concurrent.LinkedTransferQueue;
import java.util.concurrent.Semaphore;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicInteger;
Expand Down Expand Up @@ -751,7 +751,7 @@ protected final ShardSearchRequest buildShardSearchRequest(SearchShardIterator s

private static final class PendingExecutions {
private final Semaphore semaphore;
private final LinkedTransferQueue<Consumer<Releasable>> queue = new LinkedTransferQueue<>();
private final ConcurrentLinkedQueue<Consumer<Releasable>> queue = new ConcurrentLinkedQueue<>();

PendingExecutions(int permits) {
assert permits > 0 : "not enough permits: " + permits;
Expand All @@ -770,11 +770,10 @@ void submit(Consumer<Releasable> task) {
}
}
}

}

private void executeAndRelease(Consumer<Releasable> task) {
while (task != null) {
do {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not strictly necessary but I figured it'd be nice to make it obvious that we never work on null here when entering the loop :)

final SubscribableListener<Void> onDone = new SubscribableListener<>();
task.accept(() -> onDone.onResponse(null));
if (onDone.isDone()) {
Expand All @@ -797,13 +796,21 @@ public void onFailure(Exception e) {
});
return;
}
}
} while (task != null);
}

private Consumer<Releasable> pollNextTaskOrReleasePermit() {
var task = queue.poll();
if (task == null) {
semaphore.release();
while (queue.peek() != null && semaphore.tryAcquire()) {
task = queue.poll();
if (task == null) {
semaphore.release();
} else {
return task;
}
}
}
return task;
}
Expand Down