Server-generated column statistics

[niloc132]

Reintroduces the column statistics feature from DHE. This code is copied, with a few changes to how it behaves and how it functions:

• Results are sent to the client in a static table rather than a simple object payload.
• Client can control the max number of unique values to display, up to a point (previously the max was technically 19, new default is 20).
• Non-Comparable objects now have their "popularity" counts as well as Comparable, until a max limit is reached. Only the most common 20 entries will be sent to the client.
• Deephaven "null" values are used to represent that a standard deviation or average cannot be computed.

Note that until #188 is solved, the JS API and web UI will not display the unique value list.

[devinrsmith]

Proto changes look structurally sound; but I have a larger question. Are we fundamentally performing computations here that can't be performed with our existing APIs / table operations? If the answer is "yes", would it be better to try and improve our fundamental operations to handle this use case? ColumnStatisticsRequest looks like a heavy hammer, and I'm sad to see the bespoke logic in server/src/main/java/io/deephaven/server/table/stats/. (I understand this is mostly a copy of DHE code, but worth raising Qs IMO.)

[chipkent]

Does this need implementation?

[niloc132]

No - every generated implementation looks like this. If you were to implement a python server, you would override this method and offer a real implementation.

[niloc132]

@devinrsmith

Are we fundamentally performing computations here that can't be performed with our existing APIs / table operations?

Yes, and deliberately so - we do not have a "Count" operation that only counts non-nulls, and we do not have a "unique" operation that only selects the N most unique. Neither is impossible to add, but @rcaudy specifically prefers that for now we just stick to a specialized implementation like DHE has.

As one alternative, we could implement the two above aggregations, which would give us these same operations, except now ticking - however, the only consumer of this API is the web UI's column stats popup, which doesn't expect a Table instance, and so doesn't support ticking data. So the only win here would be that we would implement two new aggregation types, and then not need the rest of this new code.

[rcaudy]

This is a review of everything but the JS* classes.

[rcaudy]

@devinrsmith

Are we fundamentally performing computations here that can't be performed with our existing APIs / table operations?

Yes, and deliberately so - we do not have a "Count" operation that only counts non-nulls, and we do not have a "unique" operation that only selects the N most unique. Neither is impossible to add, but @rcaudy specifically prefers that for now we just stick to a specialized implementation like DHE has.

As one alternative, we could implement the two above aggregations, which would give us these same operations, except now ticking - however, the only consumer of this API is the web UI's column stats popup, which doesn't expect a Table instance, and so doesn't support ticking data. So the only win here would be that we would implement two new aggregation types, and then not need the rest of this new code.

We discussed this at some length before doing the porting. I would prefer to use our aggs, and thus standardize on one set of code for this. What convinced me that we should just port is:

1. The client cannot consume ticking stats (if it could, I would insist we use aggs)
2. This code can be "tighter", only scanning each chunk a single time
3. We can ship faster this way

[rcaudy]

.

[rcaudy]

One minor bug, and some code cleanup suggestions.