Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable the Panama Vector module #96453

Merged
merged 11 commits into from
Jun 1, 2023
4 changes: 4 additions & 0 deletions distribution/src/config/jvm.options
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,10 @@
## JVM temporary directory
-Djava.io.tmpdir=${ES_TMPDIR}

# Leverages accelerated vector hardware instructions; removing this may
# result in less optimal vector performance
20:--add-modules=jdk.incubator.vector

## heap dumps

# generate a heap dump when an allocation from the Java heap fails; heap dumps
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
import java.io.InputStreamReader;
import java.io.PrintWriter;
import java.nio.charset.StandardCharsets;
import java.util.List;
import java.util.concurrent.CountDownLatch;

import static org.elasticsearch.bootstrap.BootstrapInfo.SERVER_READY_MARKER;
Expand Down Expand Up @@ -68,6 +69,9 @@ void drain() {
nonInterruptibleVoid(this::join);
}

/** List of messages / lines to filter from the output. */
List<String> filter = List.of("WARNING: Using incubator modules: jdk.incubator.vector");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can this be moved to the top of the class and made private static final?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, only seen this after merging. I'll move this as a follow-up.


@Override
public void run() {
try {
Expand All @@ -76,7 +80,7 @@ public void run() {
if (line.isEmpty() == false && line.charAt(0) == SERVER_READY_MARKER) {
ready = true;
readyOrDead.countDown();
} else {
} else if (filter.contains(line) == false) {
ChrisHegarty marked this conversation as resolved.
Show resolved Hide resolved
writer.println(line);
}
}
Expand Down
5 changes: 5 additions & 0 deletions docs/changelog/96453.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
pr: 96453
summary: Leverage SIMD hardware instructions in Vector Search
area: Vector Search
type: enhancement
issues: [96370]
Original file line number Diff line number Diff line change
Expand Up @@ -269,8 +269,9 @@ public FileVisitResult visitFile(final Path file, final BasicFileAttributes attr
LogManager.getLogger("stderr"),
Level.WARN,
// MMapDirectory messages come from Lucene, suggesting to users as a warning that they should enable preview features in
// the JDK
List.of("MMapDirectory")
// the JDK. Vector logs come from Lucene too, but only if the used explicitly disables the Vector API - no point warning
// in this case.
List.of("MMapDirectory", "VectorUtilProvider", "WARNING: Java vector incubator module is not readable")
),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This suppresses the two line warning from Lucene, if one was to disable the Vector API (by editing jvm.options).

[2023-06-01T15:15:29,430][WARN ][stderr                   ] [yamlRestTest-0] Jun 01, 2023 3:15:29 PM org.apache.lucene.util.VectorUtilProvider lookup
[2023-06-01T15:15:29,434][WARN ][stderr                   ] [yamlRestTest-0] WARNING: Java vector incubator module is not readable. For optimal vector performance, pass '--add-modules jdk.incubator.vector' to enable Vector API.

false,
StandardCharsets.UTF_8
Expand Down