Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add value ranges for GeoTile aggregation metrics metrics in the meta layer #71611

Merged
merged 7 commits into from
Apr 22, 2021

Conversation

iverase
Copy link
Contributor

@iverase iverase commented Apr 13, 2021

In order to support automatic styling, the consumers of vector tiles need to know the range of values added to the aggs layer. Therefore this PR adds this values ranges on the meta later. Values are computed using min / max pipeline aggregations.

So for example, there is the document counts range in the form:

_count.min
_count.max

In addition, for any metric aggregation there will be two entries:

aggs.<aggName>.min
aggs.<aggName>.max

@iverase iverase added the :Analytics/Geo Indexing, search aggregations of geo points and shapes label Apr 13, 2021
@iverase iverase requested a review from imotov April 13, 2021 08:12
@elasticmachine elasticmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Apr 13, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-analytics-geo (Team:Analytics)

Copy link
Contributor

@imotov imotov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's get the bug-fix in, but let's wait with handling aggregations here. I think I have a more generic solution to this one.

@@ -226,9 +234,31 @@ public void setSize(int size) {
}

public void setAggBuilder(AggregatorFactories.Builder aggBuilder) {
// TODO: validation
for (AggregationBuilder aggregation : aggBuilder.getAggregatorFactories()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's wait with this. I think I have some idea on how to make this more generic.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still wonder if 1) this should responsibility of the callers and 2) if we want to add these pipeline aggs, we need to throw an exception if we don't recognize the agg

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My hope is that a user of this API does not need to be an expert on Elasticsearch so we can provide this information from them. I am hoping to only support a subset of the aggregations, the ones that make sense.

@@ -57,10 +57,10 @@ public void box(VectorTile.Tile.Feature.Builder featureBuilder, double minLon, d
}

private int lat(double lat) {
return (int) (pointYScale * (VectorTileUtils.latToSphericalMercator(lat) - rectangle.getMinY())) + extent;
return (int) Math.round(pointYScale * (VectorTileUtils.latToSphericalMercator(lat) - rectangle.getMinY())) + extent;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should still probably get this in.

@iverase
Copy link
Contributor Author

iverase commented Apr 20, 2021

Thanks @imotov , I merged your changes into this PR, and the output of the pipeline aggregations are now added to the meta layer using your approach (very cool)

@imotov
Copy link
Contributor

imotov commented Apr 20, 2021

Nice!

// top term and percentile should be supported
throw new IllegalArgumentException("Unknown feature type [" + type + "]");
}
addToXContentToFeature(featureBuilder, layerProps, aggregation);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add the result of aggregations from the Content as well

@iverase
Copy link
Contributor Author

iverase commented Apr 21, 2021

I moved the methods that add properties to the layer to a hyper class and rename some of the methods. @imotov should be push this PR?

Copy link
Contributor

@imotov imotov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's get this in, but I would love to iterate on it a bit more next. Something bothers me about this case statement with hardcoded list of 5 aggregations. I feel like it really can be any NumericMetricsAggregator.SingleValue derived agg there.

}
}
for (PipelineAggregationBuilder aggregation : aggBuilder.getPipelineAggregatorFactories()) {
// should not have pipeline aggregations
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why?

@iverase iverase merged commit d6cfec0 into elastic:feature/vector-tiles Apr 22, 2021
@iverase iverase deleted the rangesMetaLayer branch April 22, 2021 05:50
@iverase
Copy link
Contributor Author

iverase commented Apr 22, 2021

I feel like it really can be any NumericMetricsAggregator.SingleValue derived agg there.

Yes but I think there is nothing in the aggregation builder that tells you the type of aggregation?

@imotov
Copy link
Contributor

imotov commented Apr 22, 2021

Indeed, we need to figure this part out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/Geo Indexing, search aggregations of geo points and shapes Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants