Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Server-generated column statistics #4680

Merged
merged 39 commits into from
Nov 2, 2023
Merged
Show file tree
Hide file tree
Changes from 38 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
86fe227
gRPC wiring for column statistics request
niloc132 Oct 6, 2023
d5d23e7
Regenerate GWT Bindings
niloc132 Oct 6, 2023
815c3c7
Stub in JS API wiring
niloc132 Oct 6, 2023
bdd119e
Check in py/cpp/go generated types
niloc132 Oct 6, 2023
9af8d69
Basically functional replication, almost specialized for non-Comparable
niloc132 Oct 13, 2023
4a7feaa
Specialize for Comparable, actually replicated impls
niloc132 Oct 13, 2023
992e3b5
Rewrite impl to use Table instead of Result, spotless
niloc132 Oct 16, 2023
ef4c568
Integrate with js api
niloc132 Oct 17, 2023
465bb02
Nearly-working js calls
niloc132 Oct 17, 2023
d924fff
draft to drop col
niloc132 Oct 18, 2023
457270e
consistently use null instead of nan/infinity for avg/stddev
niloc132 Oct 18, 2023
e144389
correctly drop unreadable columns
niloc132 Oct 18, 2023
7b886d6
Quick draft at non-Comparable impl for object stats
niloc132 Oct 18, 2023
e3ef773
Self review, lots of cleanup
niloc132 Oct 20, 2023
3fd396e
Update generated py/cpp protos
niloc132 Oct 20, 2023
963b038
Fix stddev math, make result table column names consistent
niloc132 Oct 20, 2023
eff4f39
Style/naming/docs
niloc132 Oct 20, 2023
9922ac1
Update cpp/py proto
niloc132 Oct 20, 2023
f624e5b
Update gwt proto bindings
niloc132 Oct 20, 2023
112fd06
Add a test, and correct bugs demonstrated by that test
niloc132 Oct 20, 2023
aff3fed
Update server/src/main/java/io/deephaven/server/table/ops/ColumnStati…
niloc132 Oct 27, 2023
e522f08
Apply suggestions from code review
niloc132 Oct 27, 2023
f6b265a
Review feedback
niloc132 Oct 27, 2023
945ffd9
Update generated proto files
niloc132 Oct 27, 2023
cff46fb
WIP review feedback
niloc132 Oct 27, 2023
c62aa2e
Ensure defaults for float/double generation result in random values
niloc132 Oct 27, 2023
e17dc73
Merge branch 'nightly/test-data-generator-fixes' into 697-column-stats-2
niloc132 Oct 27, 2023
d304811
Review feedback
niloc132 Oct 31, 2023
5cc40af
review feedback
niloc132 Oct 31, 2023
17960eb
char-specific data structures
niloc132 Oct 31, 2023
92c4a91
slightly better codegen
niloc132 Oct 31, 2023
dd51367
More concise datetime chunked stats
niloc132 Oct 31, 2023
97b4385
spotless
niloc132 Oct 31, 2023
1a94bd9
Import cleanup, remove unnecessary hasNext
niloc132 Oct 31, 2023
562782b
Merge branch 'main' into 697-column-stats-2
niloc132 Nov 1, 2023
1b748f2
Cosmetic changes for future readers
niloc132 Nov 1, 2023
8028f30
Correct range for test
niloc132 Nov 1, 2023
b0b453f
Refine stddev precision
niloc132 Nov 1, 2023
2caa890
Review feedback, addit tests
niloc132 Nov 2, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -92,11 +92,9 @@ public void onCallStartedMessageStream(AuthContext authContext) {}

public void onMessageReceivedMessageStream(AuthContext authContext, StreamRequest request) {}

public void onMessageReceivedOpenMessageStream(AuthContext authContext,
StreamRequest request) {}
public void onMessageReceivedOpenMessageStream(AuthContext authContext, StreamRequest request) {}

public void onMessageReceivedNextMessageStream(AuthContext authContext,
StreamRequest request) {}
public void onMessageReceivedNextMessageStream(AuthContext authContext, StreamRequest request) {}
}

class DenyAll implements ObjectServiceAuthWiring {
Expand All @@ -112,13 +110,11 @@ public void onMessageReceivedMessageStream(AuthContext authContext, StreamReques
ServiceAuthWiring.operationNotAllowed();
}

public void onMessageReceivedOpenMessageStream(AuthContext authContext,
StreamRequest request) {
public void onMessageReceivedOpenMessageStream(AuthContext authContext, StreamRequest request) {
ServiceAuthWiring.operationNotAllowed();
}

public void onMessageReceivedNextMessageStream(AuthContext authContext,
StreamRequest request) {
public void onMessageReceivedNextMessageStream(AuthContext authContext, StreamRequest request) {
ServiceAuthWiring.operationNotAllowed();
}
}
Expand All @@ -144,15 +140,13 @@ public void onMessageReceivedMessageStream(AuthContext authContext, StreamReques
}
}

public void onMessageReceivedOpenMessageStream(AuthContext authContext,
StreamRequest request) {
public void onMessageReceivedOpenMessageStream(AuthContext authContext, StreamRequest request) {
if (delegate != null) {
delegate.onMessageReceivedOpenMessageStream(authContext, request);
}
}

public void onMessageReceivedNextMessageStream(AuthContext authContext,
StreamRequest request) {
public void onMessageReceivedNextMessageStream(AuthContext authContext, StreamRequest request) {
if (delegate != null) {
delegate.onMessageReceivedNextMessageStream(authContext, request);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
import io.deephaven.proto.backplane.grpc.AjRajTablesRequest;
import io.deephaven.proto.backplane.grpc.ApplyPreviewColumnsRequest;
import io.deephaven.proto.backplane.grpc.AsOfJoinTablesRequest;
import io.deephaven.proto.backplane.grpc.ColumnStatisticsRequest;
import io.deephaven.proto.backplane.grpc.ComboAggregateRequest;
import io.deephaven.proto.backplane.grpc.CreateInputTableRequest;
import io.deephaven.proto.backplane.grpc.CrossJoinTablesRequest;
Expand Down Expand Up @@ -514,6 +515,17 @@ void checkPermissionSeekRow(AuthContext authContext, SeekRowRequest request,
void checkPermissionMetaTable(AuthContext authContext, MetaTableRequest request,
List<Table> sourceTables);

/**
* Authorize a request to ComputeColumnStatistics.
*
* @param authContext the authentication context of the request
* @param request the request to authorize
* @param sourceTables the operation's source tables
* @throws io.grpc.StatusRuntimeException if the user is not authorized to invoke ComputeColumnStatistics
*/
void checkPermissionComputeColumnStatistics(AuthContext authContext,
ColumnStatisticsRequest request, List<Table> sourceTables);

/**
* A default implementation that funnels all requests to invoke {@code checkPermission}.
*/
Expand Down Expand Up @@ -729,6 +741,11 @@ public void checkPermissionMetaTable(AuthContext authContext, MetaTableRequest r
List<Table> sourceTables) {
checkPermission(authContext, sourceTables);
}

public void checkPermissionComputeColumnStatistics(AuthContext authContext,
ColumnStatisticsRequest request, List<Table> sourceTables) {
checkPermission(authContext, sourceTables);
}
}

/**
Expand Down Expand Up @@ -1045,5 +1062,12 @@ public void checkPermissionMetaTable(AuthContext authContext, MetaTableRequest r
delegate.checkPermissionMetaTable(authContext, request, sourceTables);
}
}

public void checkPermissionComputeColumnStatistics(AuthContext authContext,
ColumnStatisticsRequest request, List<Table> sourceTables) {
if (delegate != null) {
delegate.checkPermissionComputeColumnStatistics(authContext, request, sourceTables);
}
}
}
}

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading