Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove Adjacency_matrix setting #46327

Merged
merged 1 commit into from
Sep 19, 2019
Merged

Conversation

markharwood
Copy link
Contributor

Now that PR #46257 has improved internal memory usage we no longer need the default limit of 100 filters defined by index.max_adjacency_matrix_filters. This PR removes the setting and uses the existing indices.query.bool.max_clause_count setting for Boolean clause limits to apply to the number of filters used in an adjacency matrix.

Closes #46324

@markharwood markharwood self-assigned this Sep 4, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-analytics-geo

@markharwood
Copy link
Contributor Author

@jpountz this is the removal of the setting in 8.0 we talked about. Are there additional BWC things I need to add relating to dealing with mixed version clusters or opening old indices?

@@ -198,13 +198,13 @@ public String separator() {
@Override
protected AggregatorFactory doBuild(SearchContext context, AggregatorFactory parent, Builder subFactoriesBuilder)
throws IOException {
int maxFilters = context.indexShard().indexSettings().getMaxAdjacencyMatrixFilters();
int maxFilters = SearchModule.INDICES_MAX_CLAUSE_COUNT_SETTING.get(context.indexShard().indexSettings().getSettings());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is correct since you are looking up a node setting in the index settings?

Copy link
Contributor Author

@markharwood markharwood Sep 5, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UPDATE: @jpountz I got this from QueryParserHelper and it looks like that would have this issue.

However, testing it with an abusive query string/index shows that it does pick up the correct setting from the elasticsearch.yml file so not sure how that magic works its way through.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Presumably we just substitute for BooleanQuery.getMaxClauseCount() ?

Copy link
Contributor Author

@markharwood markharwood Sep 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jpountz I updated the PR to use BooleanQuery.getMaxClauseCount() as the source of truth. Look good to you?
@romseygeek thought that would be the right thing to do.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@markharwood
Copy link
Contributor Author

markharwood commented Sep 12, 2019

(Answering my own question here:) indices created in 7.x with the index.max_adjacency_matrix_filters setting set can be opened in 8.x - the setting is just moved into an "archived" part of the settings rather than throwing any error.

@markharwood markharwood force-pushed the fix/46324 branch 6 times, most recently from 784a664 to f718cb5 Compare September 18, 2019 17:27
@markharwood markharwood merged commit dc0abec into elastic:master Sep 19, 2019
nik9000 added a commit to nik9000/elasticsearch that referenced this pull request Sep 15, 2021
We dropped the adjacency matrix max clause setting in elastic#46327 because we
believe that it is no longer required. From here on out you can use the
regular boolean max clause setting.
nik9000 added a commit that referenced this pull request Sep 22, 2021
We dropped the adjacency matrix max clause setting in #46327 because we
believe that it is no longer required. From here on out you can use the
regular boolean max clause setting.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Remove redundant setting for adjacency_matrix aggregation
4 participants