Enable the Panama Vector module #96453

ChrisHegarty · 2023-05-31T09:11:43Z

This change adds the jdk.incubator.vector module, so that we can enable the Panamaized vector utils in Lucene. apache/lucene#12311

The module is added by default if running on JDK 20, which is the only current supported implementation in Lucene, but JDK 21 will likely come soon.

If for some reason this needs to be disable, just remove or otherwise comment out the --add-modules=jdk.incubator.vector line in the jvm.options.

The log output from Lucene shows the preferred vector bit width that is in operation.

relates #96370

distribution/tools/server-cli/src/main/java/org/elasticsearch/server/cli/ServerProcess.java

elasticsearchmachine · 2023-06-01T11:51:23Z

Pinging @elastic/es-search (Team:Search)

rjernst

In addition to my comment on how to plumb the option, I have a thought on the warning:

The resolution of the vector module will raise a warning on startup: WARNING: Using incubator modules: jdk.incubator.vector. This warning and the property will be documented, as well the log output from Lucene regarding the vector bit width.

For the mmap warning, we filter the response, and I think we should here too so as not to confuse users (and it won't be necessary to document specially then?). Though the way we currently filter is a bit hacky (raw stderr filtering). See #90526 (comment), which should be fixable after we implement #94613.

distribution/tools/server-cli/src/main/java/org/elasticsearch/server/cli/ServerProcess.java

elasticsearchmachine · 2023-06-01T14:20:22Z

Hi @ChrisHegarty, I've created a changelog YAML for you.

…e_vector_module

ChrisHegarty · 2023-06-01T14:54:43Z

In addition to my comment on how to plumb the option, I have a thought on the warning:

The resolution of the vector module will raise a warning on startup: WARNING: Using incubator modules: jdk.incubator.vector. This warning and the property will be documented, as well the log output from Lucene regarding the vector bit width.

For the mmap warning, we filter the response, and I think we should here too so as not to confuse users (and it won't be necessary to document specially then?). Though the way we currently filter is a bit hacky (raw stderr filtering). See #90526 (comment), which should be fixable after we implement #94613.

The incubator warning goes to stderr very early in startup. I think before we can redirect. E.g.

$ java --add-modules=jdk.incubator.vector -version 1>out 2>error
$ cat error
WARNING: Using incubator modules: jdk.incubator.vector
java version "19" 2022-09-20
Java(TM) SE Runtime Environment (build 19+36-2238)
Java HotSpot(TM) 64-Bit Server VM (build 19+36-2238, mixed mode, sharing)

This does not help, e.g.

--- a/server/src/main/java/org/elasticsearch/common/logging/LogConfigurator.java
+++ b/server/src/main/java/org/elasticsearch/common/logging/LogConfigurator.java
@@ -261,7 +261,7 @@ public class LogConfigurator {
         // third party libraries may do that. Note that we do NOT close the streams because other code may have
         // grabbed a handle to the streams and intend to write to it, eg log4j for writing to the console
         System.setOut(
-            new PrintStream(new LoggingOutputStream(LogManager.getLogger("stdout"), Level.INFO, List.of()), false, StandardCharsets.UTF_8)
+            new PrintStream(new LoggingOutputStream(LogManager.getLogger("stdout"), Level.INFO, List.of("WARNING: Using incubator modules: jdk.incubator.vector")), false, StandardCharsets.UTF_8)
         );
         System.setErr(
             new PrintStream(
@@ -270,7 +270,7 @@ public class LogConfigurator {
                     Level.WARN,
                     // MMapDirectory messages come from Lucene, suggesting to users as a warning that they should enable preview features in
                     // the JDK
-                    List.of("MMapDirectory")
+                    List.of("MMapDirectory", "WARNING: Using incubator modules: jdk.incubator.vector")
                 ),
                 false,
                 StandardCharsets.UTF_8

Or is there some way of filtering through the connection with the startup CLI process ?

ChrisHegarty · 2023-06-01T15:13:50Z

Note: Lucene emits a log message - which we absolutely want - noting the preferred bit size that is in use for the hardware that we're running on. E.g. running on my mac I can see the following in the Elasticsearch logs:

[2023-06-01T16:01:26,742][WARN ][stderr                   ] [yamlRestTest-0] Jun 01, 2023 4:01:26 PM org.apache.lucene.util.VectorUtilPanamaProvider <init>
[2023-06-01T16:01:26,742][WARN ][stderr                   ] [yamlRestTest-0] INFO: Java vector incubator API enabled; uses preferredBitSize=128

I can confirm that this is currently the case.

javanna · 2023-06-01T15:52:15Z

Thanks @ChrisHegarty ! As a follow-up, do we need to do anything on the mappings side to try and suggest / adjust the number of dimensions using even numbers that are friendly to the accelerated vector hardware instructions?

ChrisHegarty · 2023-06-01T16:27:41Z

This magic filters out the JDK's incubator warning from the output/logs, 3239e33

ChrisHegarty · 2023-06-01T16:33:11Z

Thanks @ChrisHegarty ! As a follow-up, do we need to do anything on the mappings side to try and suggest / adjust the number of dimensions using even numbers that are friendly to the accelerated vector hardware instructions?

There are certainly some dimensions that perform better than others, but of course all dimension sizes behave correctly. E.g. 1536 will perform better than 1535.

I added a note about this here: #96370 (comment)

elasticsearchmachine · 2023-06-01T17:13:46Z

Hi @ChrisHegarty, I've updated the changelog YAML for you.

…e_vector_module

ChrisHegarty · 2023-06-01T18:35:29Z

server/src/main/java/org/elasticsearch/common/logging/LogConfigurator.java

-                    List.of("MMapDirectory")
+                    // the JDK. Vector logs come from Lucene too, but only if the used explicitly disables the Vector API - no point warning
+                    // in this case.
+                    List.of("MMapDirectory", "VectorUtilProvider", "WARNING: Java vector incubator module is not readable")
                ),


This suppresses the two line warning from Lucene, if one was to disable the Vector API (by editing jvm.options).

[2023-06-01T15:15:29,430][WARN ][stderr ] [yamlRestTest-0] Jun 01, 2023 3:15:29 PM org.apache.lucene.util.VectorUtilProvider lookup [2023-06-01T15:15:29,434][WARN ][stderr ] [yamlRestTest-0] WARNING: Java vector incubator module is not readable. For optimal vector performance, pass '--add-modules jdk.incubator.vector' to enable Vector API.

distribution/tools/server-cli/src/main/java/org/elasticsearch/server/cli/ErrorPumpThread.java

rjernst

LGTM

rjernst · 2023-06-01T22:44:10Z

distribution/tools/server-cli/src/main/java/org/elasticsearch/server/cli/ErrorPumpThread.java

@@ -68,6 +69,9 @@ void drain() {
        nonInterruptibleVoid(this::join);
    }

+    /** List of messages / lines to filter from the output. */
+    List<String> filter = List.of("WARNING: Using incubator modules: jdk.incubator.vector");


nit: can this be moved to the top of the class and made private static final?

Sorry, only seen this after merging. I'll move this as a follow-up.

rjernst

LGTM

rjernst

LGTM

just turn it on and see what breaks

3e05d35

ChrisHegarty added WIP Team:Search Meta label for search team :Search Relevance/Vectors Vector search labels May 31, 2023

elasticsearchmachine added the v8.9.0 label May 31, 2023

switch based on a property.

99d3426

ChrisHegarty marked this pull request as ready for review June 1, 2023 11:48

ChrisHegarty commented Jun 1, 2023

View reviewed changes

distribution/tools/server-cli/src/main/java/org/elasticsearch/server/cli/ServerProcess.java Outdated Show resolved Hide resolved

ChrisHegarty commented Jun 1, 2023

View reviewed changes

distribution/tools/server-cli/src/main/java/org/elasticsearch/server/cli/ServerProcess.java Outdated Show resolved Hide resolved

ChrisHegarty removed the WIP label Jun 1, 2023

rjernst reviewed Jun 1, 2023

View reviewed changes

distribution/tools/server-cli/src/main/java/org/elasticsearch/server/cli/ServerProcess.java Outdated Show resolved Hide resolved

ChrisHegarty added the >enhancement label Jun 1, 2023

Update docs/changelog/96453.yaml

af0dd46

ChrisHegarty added 2 commits June 1, 2023 15:32

use jvm.options

eae28f4

Merge remote-tracking branch 'origin/enable_vector_module' into enabl…

d33194d

…e_vector_module

update change log

754e675

filter out incubator module warning.

3239e33

ChrisHegarty mentioned this pull request Jun 1, 2023

Leverage accelerated vector hardware instructions in Vector Search #96370

Closed

8 tasks

ChrisHegarty and others added 4 commits June 1, 2023 18:13

Update docs/changelog/96453.yaml

b1007eb

suppress warning if disabled

2bdc95a

Merge remote-tracking branch 'origin/enable_vector_module' into enabl…

40440cc

…e_vector_module

changelog

2d1300a

ChrisHegarty requested a review from rjernst June 1, 2023 18:32

ChrisHegarty commented Jun 1, 2023

View reviewed changes

distribution/tools/server-cli/src/main/java/org/elasticsearch/server/cli/ErrorPumpThread.java Show resolved Hide resolved

rjernst approved these changes Jun 1, 2023

View reviewed changes

ChrisHegarty merged commit 9a89ea7 into elastic:main Jun 1, 2023

ChrisHegarty deleted the enable_vector_module branch June 1, 2023 23:09

ChrisHegarty mentioned this pull request Jun 2, 2023

Change default Java Util Logging format #96519

Closed

b-deam mentioned this pull request Jun 5, 2023

Include Panama Vector module in 'jvm.options' elastic/rally-teams#79

Merged

javanna added the release highlight label Jun 9, 2023

pgomulka mentioned this pull request Jun 12, 2023

Add JUL bridge #96683

Merged

quux00 added v8.10.0 and removed v8.10.0 labels Aug 16, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable the Panama Vector module #96453

Enable the Panama Vector module #96453

ChrisHegarty commented May 31, 2023 •

edited

Loading

elasticsearchmachine commented Jun 1, 2023

rjernst left a comment

elasticsearchmachine commented Jun 1, 2023

ChrisHegarty commented Jun 1, 2023 •

edited

Loading

ChrisHegarty commented Jun 1, 2023

javanna commented Jun 1, 2023

ChrisHegarty commented Jun 1, 2023

ChrisHegarty commented Jun 1, 2023 •

edited

Loading

elasticsearchmachine commented Jun 1, 2023

ChrisHegarty Jun 1, 2023

rjernst left a comment

rjernst Jun 1, 2023

ChrisHegarty Jun 2, 2023

rjernst left a comment

rjernst left a comment

Enable the Panama Vector module #96453

Enable the Panama Vector module #96453

Conversation

ChrisHegarty commented May 31, 2023 • edited Loading

elasticsearchmachine commented Jun 1, 2023

rjernst left a comment

Choose a reason for hiding this comment

elasticsearchmachine commented Jun 1, 2023

ChrisHegarty commented Jun 1, 2023 • edited Loading

ChrisHegarty commented Jun 1, 2023

javanna commented Jun 1, 2023

ChrisHegarty commented Jun 1, 2023

ChrisHegarty commented Jun 1, 2023 • edited Loading

elasticsearchmachine commented Jun 1, 2023

ChrisHegarty Jun 1, 2023

Choose a reason for hiding this comment

rjernst left a comment

Choose a reason for hiding this comment

rjernst Jun 1, 2023

Choose a reason for hiding this comment

ChrisHegarty Jun 2, 2023

Choose a reason for hiding this comment

rjernst left a comment

Choose a reason for hiding this comment

rjernst left a comment

Choose a reason for hiding this comment

ChrisHegarty commented May 31, 2023 •

edited

Loading

ChrisHegarty commented Jun 1, 2023 •

edited

Loading

ChrisHegarty commented Jun 1, 2023 •

edited

Loading