Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] SearchQueryIT testAllDocsQueryString failing #115728

Closed
elasticsearchmachine opened this issue Oct 27, 2024 · 4 comments · Fixed by #117431
Closed

[CI] SearchQueryIT testAllDocsQueryString failing #115728

elasticsearchmachine opened this issue Oct 27, 2024 · 4 comments · Fixed by #117431
Assignees
Labels
needs:risk Requires assignment of a risk label (low, medium, blocker) :Search Foundations/Search Catch all for Search Foundations Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch >test-failure Triaged test failures from CI

Comments

@elasticsearchmachine
Copy link
Collaborator

elasticsearchmachine commented Oct 27, 2024

Build Scans:

Reproduction Line:

./gradlew ":server:internalClusterTest" --tests "org.elasticsearch.search.query.SearchQueryIT.testAllDocsQueryString" -Dtests.seed=25AA71D6AED92FE7 -Dtests.locale=mgo-Latn-CM -Dtests.timezone=Europe/Zaporozhye -Druntime.java=23

Applicable branches:
main

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.Exception: Test abandoned because suite timeout was reached.

Issue Reasons:

  • [main] 3 failures in test testAllDocsQueryString (0.5% fail rate in 605 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

@elasticsearchmachine elasticsearchmachine added :Search/Search Search-related issues that do not fall into other categories >test-failure Triaged test failures from CI Team:Search Meta label for search team needs:risk Requires assignment of a risk label (low, medium, blocker) labels Oct 27, 2024
@elasticsearchmachine
Copy link
Collaborator Author

Pinging @elastic/es-search (Team:Search)

@benwtrent
Copy link
Member

OK, failed twice due to suite timeout. Both of these were Linux.

Interestingly, this was part of the ERROR output of the test:

1> [2024-10-27T21:18:55,037][INFO ][o.e.s.q.SearchQueryIT    ] [testAllDocsQueryString] before test |  
  | 1> [2024-10-27T21:18:55,038][INFO ][o.e.s.q.SearchQueryIT    ] [testAllDocsQueryString] [SearchQueryIT#testAllDocsQueryString]: setting up test |  
  | 1> [2024-10-27T21:18:55,042][INFO ][o.e.c.m.MetadataIndexTemplateService] [node_s0] adding template [random_index_template] for index patterns [*] |  
  | 1> [2024-10-27T21:18:55,051][INFO ][o.e.s.q.SearchQueryIT    ] [testAllDocsQueryString] [SearchQueryIT#testAllDocsQueryString]: all set up test |  
  | 1> [2024-10-27T21:18:55,054][INFO ][o.e.c.m.MetadataCreateIndexService] [node_s0] [test] creating index, cause [api], templates [random_index_template], shards [4]/[2] |  
  | 1> [2024-10-27T21:18:55,103][INFO ][o.e.s.q.SearchQueryIT    ] [testAllDocsQueryString] Index [4] docs async: [true] bulk: [false] |  
  | 1> [2024-10-27T21:18:55,117][INFO ][o.e.c.m.MetadataMappingService] [node_s0] [test/NWUCp2YAQN-CaWqRps31ZQ] create_mapping |  
  | 1> [2024-10-27T21:18:55,118][INFO ][o.e.c.m.MetadataMappingService] [node_s0] [test/NWUCp2YAQN-CaWqRps31ZQ] update_mapping [_doc] |  
  | 1> [2024-10-27T21:18:55,177][INFO ][o.e.c.r.a.AllocationService] [node_s0] current.health="GREEN" message="Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[test][1]]])." previous.health="YELLOW" reason="shards started [[test][1]]" |  
  | 1> [2024-10-27T21:22:26,757][ERROR][o.e.t.LeakTracker        ] [[Cleaner-0]] LEAK: resource was not cleaned up before it was garbage-collected.

<br class="Apple-interchange-newline">message="Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[test][1]]])." previous.health="YELLOW" reason="shards started [[test][1]]" |  
-- | --
  | 1> [2024-10-27T21:22:26,757][ERROR][o.e.t.LeakTracker        ] [[Cleaner-0]] LEAK: resource was not cleaned up before it was garbage-collected. |  
  | 1> Recent access records: |  
  | 1> Created at: |  
  | 1> 	in [TEST-SearchQueryIT.testAllDocsQueryString-seed#[99F013EBD9BC8D98]][testAllDocsQueryString] |  
  | 1> 	org.elasticsearch.action.search.ArraySearchPhaseResults.<init>(ArraySearchPhaseResults.java:28) |  
  | 1> 	org.elasticsearch.action.search.QueryPhaseResultConsumer.<init>(QueryPhaseResultConsumer.java:85) |  
  | 1> 	org.elasticsearch.action.search.SearchPhaseController.newSearchPhaseResults(SearchPhaseController.java:824) |  
  | 1> 	org.elasticsearch.action.search.TransportSearchAction$AsyncSearchActionProvider.newSearchPhase(TransportSearchAction.java:1502) |  
  | 1> 	org.elasticsearch.action.search.TransportSearchAction.executeSearch(TransportSearchAction.java:1305) |  
  | 1> 	org.elasticsearch.action.search.TransportSearchAction.executeLocalSearch(TransportSearchAction.java:1038) |  
  | 1> 	org.elasticsearch.action.search.TransportSearchAction.lambda$executeRequest$4(TransportSearchAction.java:365) |  
  | 1> 	org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247) |  
  | 1> 	org.elasticsearch.index.query.Rewriteable.rewriteAndFetch(Rewriteable.java:109) |  
  | 1> 	org.elasticsearch.index.query.Rewriteable.rewriteAndFetch(Rewriteable.java:77) |  
  | 1> 	org.elasticsearch.action.search.TransportSearchAction.executeRequest(TransportSearchAction.java:531) |  
  | 1> 	org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:316) |  
  | 1> 	org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:121) |  
  | 1> 	org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:135) |  
  | 1> 	org.elasticsearch.action.support.TransportAction.handleExecution(TransportAction.java:96) |  
  | 1> 	org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:59) |  
  | 1> 	org.elasticsearch.tasks.TaskManager.registerAndExecute(TaskManager.java:197) |  
  | 1> 	org.elasticsearch.client.internal.node.NodeClient.executeLocally(NodeClient.java:106) |  
  | 1> 	org.elasticsearch.client.internal.node.NodeClient.doExecute(NodeClient.java:84) |  
  | 1> 	org.elasticsearch.client.internal.support.AbstractClient.execute(AbstractClient.java:140) |  
  | 1> 	org.elasticsearch.client.internal.FilterClient.doExecute(FilterClient.java:56) |  
  | 1> 	org.elasticsearch.client.internal.support.AbstractClient.execute(AbstractClient.java:140) |  
  | 1> 	org.elasticsearch.client.internal.support.AbstractClient.execute(AbstractClient.java:126) |  
  | 1> 	org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:37) |  
  | 1> 	org.elasticsearch.action.ActionRequestBuilder.get(ActionRequestBuilder.java:44) |  
  | 1> 	org.elasticsearch.action.ActionRequestBuilder.get(ActionRequestBuilder.java:17) |  
  | 1> 	org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertResponse(ElasticsearchAssertions.java:382) |  
  | 1> 	org.elasticsearch.search.query.SearchQueryIT.testAllDocsQueryString(SearchQueryIT.java:301) |  
  | 1> 	java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) |  
  | 1> 	java.base/java.lang.reflect.Method.invoke(Method.java:580) |  
  | 1> 	com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) |  
  | 1> 	com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) |  
  | 1> 	com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) |  
  | 1> 	com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) |  
  | 1> 	com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) |  
  | 1> 	org.junit.rules.RunRules.evaluate(RunRules.java:20) |  
  | 1> 	org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) |  
  | 1> 	org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) |  
  | 1> 	org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) |  
  | 1> 	org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) |  
  | 1> 	org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) |  
  | 1> 	org.junit.rules.RunRules.evaluate(RunRules.java:20) |  
  | 1> 	com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) |  
  | 1> 	com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) |  
  | 1> 	com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) |  
  | 1> 	com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) |  
  | 1> 	com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) |  
  | 1> 	com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) |  
  | 1> 	com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) |  
  | 1> 	com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) |  
  | 1> 	org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) |  
  | 1> 	com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) |  
  | 1> 	org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) |  
  | 1> 	com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) |  
  | 1> 	com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) |  
  | 1> 	com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) |  
  | 1> 	com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) |  
  | 1> 	org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) |  
  | 1> 	org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) |  
  | 1> 	org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) |  
  | 1> 	org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) |  
  | 1> 	org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) |  
  | 1> 	org.junit.rules.RunRules.evaluate(RunRules.java:20) |  
  | 1> 	com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) |  
  | 1> 	com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) |  
  | 1> 	com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) |  
  | 1> 	java.base/java.lang.Thread.run(Thread.java:1570) |  
  | 1> [2024-10-27T21:22:26,758][ERROR][o.e.t.LeakTracker        ] [[Cleaner-0]] LEAK: resource was not cleaned up before it was garbage-collected. |  
  | 1> Recent access records: |  
  | 1> Created at: |  
  | 1> 	in [TEST-SearchQueryIT.testAllDocsQueryString-seed#[99F013EBD9BC8D98]][testAllDocsQueryString] |  
  | 1> 	org.elasticsearch.action.search.ArraySearchPhaseResults.<init>(ArraySearchPhaseResults.java:28) |  
  | 1> 	org.elasticsearch.action.search.SearchDfsQueryThenFetchAsyncAction.<init>(SearchDfsQueryThenFetchAsyncAction.java:70) |  
  | 1> 	org.elasticsearch.action.search.TransportSearchAction$AsyncSearchActionProvider.newSearchPhase(TransportSearchAction.java:1515) |  
  | 1> 	org.elasticsearch.action.search.TransportSearchAction.executeSearch(TransportSearchAction.java:1305) |  
  | 1> 	org.elasticsearch.action.search.TransportSearchAction.executeLocalSearch(TransportSearchAction.java:1038) |  
  | 1> 	org.elasticsearch.action.search.TransportSearchAction.lambda$executeRequest$4(TransportSearchAction.java:365) |  
  | 1> 	org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247) |  
  | 1> 	org.elasticsearch.index.query.Rewriteable.rewriteAndFetch(Rewriteable.java:109) |  
  | 1> 	org.elasticsearch.index.query.Rewriteable.rewriteAndFetch(Rewriteable.java:77) |  
  | 1> 	org.elasticsearch.action.search.TransportSearchAction.executeRequest(TransportSearchAction.java:531) |  
  | 1> 	org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:316) |  
  | 1> 	org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:121) |  
  | 1> 	org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:135) |  
  | 1> 	org.elasticsearch.action.support.TransportAction.handleExecution(TransportAction.java:96) |  
  | 1> 	org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:59) |  
  | 1> 	org.elasticsearch.tasks.TaskManager.registerAndExecute(TaskManager.java:197) |  
  | 1> 	org.elasticsearch.client.internal.node.NodeClient.executeLocally(NodeClient.java:106) |  
  | 1> 	org.elasticsearch.client.internal.node.NodeClient.doExecute(NodeClient.java:84) |  
  | 1> 	org.elasticsearch.client.internal.support.AbstractClient.execute(AbstractClient.java:140) |  
  | 1> 	org.elasticsearch.client.internal.FilterClient.doExecute(FilterClient.java:56) |  
  | 1> 	org.elasticsearch.client.internal.support.AbstractClient.execute(AbstractClient.java:140) |  
  | 1> 	org.elasticsearch.client.internal.support.AbstractClient.execute(AbstractClient.java:126) |  
  | 1> 	org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:37) |  
  | 1> 	org.elasticsearch.action.ActionRequestBuilder.get(ActionRequestBuilder.java:44) |  
  | 1> 	org.elasticsearch.action.ActionRequestBuilder.get(ActionRequestBuilder.java:17) |  
  | 1> 	org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertResponse(ElasticsearchAssertions.java:382) |  
  | 1> 	org.elasticsearch.search.query.SearchQueryIT.testAllDocsQueryString(SearchQueryIT.java:301) |  
  | 1> 	java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) |  
  | 1> 	java.base/java.lang.reflect.Method.invoke(Method.java:580) |  
  | 1> 	com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) |  
  | 1> 	com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) |  
  | 1> 	com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) |  
  | 1> 	com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) |  
  | 1> 	com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) |  
  | 1> 	org.junit.rules.RunRules.evaluate(RunRules.java:20) |  
  | 1> 	org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) |  
  | 1> 	org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) |  
  | 1> 	org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) |  
  | 1> 	org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) |  
  | 1> 	org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) |  
  | 1> 	org.junit.rules.RunRules.evaluate(RunRules.java:20) |  
  | 1> 	com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) |  
  | 1> 	com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) |  
  | 1> 	com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) |  
  | 1> 	com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) |  
  | 1> 	com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) |  
  | 1> 	com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) |  
  | 1> 	com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) |  
  | 1> 	com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) |  
  | 1> 	org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) |  
  | 1> 	com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) |  
  | 1> 	org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) |  
  | 1> 	com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) |  
  | 1> 	com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) |  
  | 1> 	com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) |  
  | 1> 	com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) |  
  | 1> 	org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) |  
  | 1> 	org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) |  
  | 1> 	org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) |  
  | 1> 	org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) |  
  | 1> 	org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) |  
  | 1> 	org.junit.rules.RunRules.evaluate(RunRules.java:20) |  
  | 1> 	com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) |  
  | 1> 	com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) |  
  | 1> 	com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) |  
  | 1> 	java.base/java.lang.Thread.run(Thread.java:1570)

@benwtrent benwtrent added :Search Foundations/Search Catch all for Search Foundations and removed :Search/Search Search-related issues that do not fall into other categories labels Oct 28, 2024
@elasticsearchmachine elasticsearchmachine added Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch and removed Team:Search Meta label for search team labels Oct 28, 2024
@elasticsearchmachine
Copy link
Collaborator Author

Pinging @elastic/es-search-foundations (Team:Search Foundations)

@elasticsearchmachine
Copy link
Collaborator Author

This has been muted on branch main

Mute Reasons:

  • [main] 3 failures in test testAllDocsQueryString (0.5% fail rate in 605 executions)

Build Scans:

jfreden pushed a commit to jfreden/elasticsearch that referenced this issue Nov 4, 2024
@original-brownbear original-brownbear self-assigned this Nov 22, 2024
elasticsearchmachine pushed a commit that referenced this issue Nov 24, 2024
This is long fixed by #116264

Fixes #115728
alexey-ivanov-es pushed a commit to alexey-ivanov-es/elasticsearch that referenced this issue Nov 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs:risk Requires assignment of a risk label (low, medium, blocker) :Search Foundations/Search Catch all for Search Foundations Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch >test-failure Triaged test failures from CI
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants