[BUG] Plugin returns NullPointerException for any query after a specific one

[Yury-Fridlyand]

What is the bug?

SQL returns NullPointerException to any query in some condition.

How can one reproduce the bug?

1. Run a valid SQL query - get a valid answer

opensearchsql> select COUNT(*) from `beer.stackexchange`;
fetched rows / total rows = 1/1
+------------+
| COUNT(*)   |
|------------|
| 60         |
+------------+

2. Run a valid PPL query - get an error

opensearchsql> source=beer.stackexchange | where multi_match([Title, Body, Tags], 'hops');
{'reason': 'Invalid Query', 'details': '', 'type': 'NullPointerException'}

Exception stack

Error happened during query handling
java.lang.NullPointerException: null
        at org.opensearch.sql.analysis.Analyzer.visitRelation(Analyzer.java:130) ~[core-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.analysis.Analyzer.visitRelation(Analyzer.java:92) ~[core-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.ast.tree.Relation.accept(Relation.java:105) ~[core-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.analysis.Analyzer.visitFilter(Analyzer.java:169) ~[core-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.analysis.Analyzer.visitFilter(Analyzer.java:92) ~[core-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.ast.tree.Filter.accept(Filter.java:44) ~[core-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.analysis.Analyzer.visitProject(Analyzer.java:288) ~[core-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.analysis.Analyzer.visitProject(Analyzer.java:92) ~[core-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.ast.tree.Project.accept(Project.java:71) ~[core-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.analysis.Analyzer.analyze(Analyzer.java:115) ~[core-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.ppl.PPLService.plan(PPLService.java:89) ~[ppl-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.ppl.PPLService.execute(PPLService.java:59) [ppl-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.plugin.transport.TransportPPLQueryAction.doExecute(TransportPPLQueryAction.java:93) [opensearch-sql-2.4.0.0-SNAPSHOT.jar:2.4.0.0-SNAPSHOT]
        at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:218) [opensearch-2.4.0-SNAPSHOT.jar:2.4.0-SNAPSHOT]
        at org.opensearch.action.support.TransportAction.execute(TransportAction.java:188) [opensearch-2.4.0-SNAPSHOT.jar:2.4.0-SNAPSHOT]
        at org.opensearch.action.support.TransportAction.execute(TransportAction.java:107) [opensearch-2.4.0-SNAPSHOT.jar:2.4.0-SNAPSHOT]
        at org.opensearch.client.node.NodeClient.executeLocally(NodeClient.java:110) [opensearch-2.4.0-SNAPSHOT.jar:2.4.0-SNAPSHOT]
        at org.opensearch.client.node.NodeClient.doExecute(NodeClient.java:97) [opensearch-2.4.0-SNAPSHOT.jar:2.4.0-SNAPSHOT]
        at org.opensearch.client.support.AbstractClient.execute(AbstractClient.java:426) [opensearch-2.4.0-SNAPSHOT.jar:2.4.0-SNAPSHOT]
        at org.opensearch.sql.plugin.rest.RestPPLQueryAction.lambda$prepareRequest$2(RestPPLQueryAction.java:117) [opensearch-sql-2.4.0.0-SNAPSHOT.jar:2.4.0.0-SNAPSHOT]
        at org.opensearch.sql.opensearch.executor.Scheduler.lambda$withCurrentContext$0(Scheduler.java:30) [opensearch-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:747) [opensearch-2.4.0-SNAPSHOT.jar:2.4.0-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]

3. Run SQL query again - get a error ❗

opensearchsql> select COUNT(*) from `beer.stackexchange`;
{'reason': 'Invalid SQL query', 'details': '', 'type': 'NullPointerException'}

Exception stack

Client side error during query execution
java.lang.NullPointerException: null
        at org.opensearch.sql.analysis.Analyzer.visitRelation(Analyzer.java:130) ~[core-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.analysis.Analyzer.visitRelation(Analyzer.java:92) ~[core-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.ast.tree.Relation.accept(Relation.java:105) ~[core-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.analysis.Analyzer.visitAggregation(Analyzer.java:212) ~[core-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.analysis.Analyzer.visitAggregation(Analyzer.java:92) ~[core-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.ast.tree.Aggregation.accept(Aggregation.java:76) ~[core-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.analysis.Analyzer.visitProject(Analyzer.java:288) ~[core-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.analysis.Analyzer.visitProject(Analyzer.java:92) ~[core-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.ast.tree.Project.accept(Project.java:71) ~[core-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.analysis.Analyzer.analyze(Analyzer.java:115) ~[core-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.sql.SQLService.analyze(SQLService.java:97) ~[sql-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.legacy.plugin.RestSQLQueryAction.prepareRequest(RestSQLQueryAction.java:110) ~[legacy-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.legacy.plugin.RestSqlAction.lambda$prepareRequest$1(RestSqlAction.java:158) [legacy-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.opensearch.executor.Scheduler.lambda$withCurrentContext$0(Scheduler.java:30) [opensearch-2.4.0.0-SNAPSHOT.jar:?]
        at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:747) [opensearch-2.4.0-SNAPSHOT.jar:2.4.0-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
        at java.lang.Thread.run(Thread.java:829) [?:?]

What is the expected behavior?

1. No error on valid queries
2. No context/data/whatever persist after a query or shared between queries

What is your host/environment?

2.x @ 10e44ee

Do you have any additional context?

Workaround - restart server

[Yury-Fridlyand]

I suspect catalog feature conflicts with dots in index/table name.
I played a lot with this index name while was working on relevance functions, it was acceptable.

[MaxKsyunz]

OPEN_SEARCH catalog is removed from CatalogServiceImpl.storageEngineMap during CatalogServiceImpl.getCatalog because removing an item from a keySet will remove the corresponding entry from the map.

[MaxKsyunz]

@penghuo @dai-chen @anirudha this made it into 2.3. Do you think it warrants a hotfix?

[dai-chen]

@penghuo @dai-chen @anirudha this made it into 2.3. Do you think it warrants a hotfix?

I think we're good. I recall we merged the PR to main branch first and then to 2.x after 2.3 release. We can double check: https://github.com/opensearch-project/sql/blob/2.3/plugin/src/main/java/org/opensearch/sql/plugin/catalog/CatalogServiceImpl.java doesn't exist in 2.3 branch.