Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend bidirectional provider by introducing sample interval filtering #141

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

rafaeling
Copy link
Contributor

No description provided.

@rafaeling rafaeling changed the title Bidirectional provider start stop Extend bidirectional provider by introducing sample interval filtering Feb 26, 2025
// Duration of the active call. If it is not set, call will least for ever.
int32 duration_ms = 1;
// Max desired sample update interval.
SampleInterval max_sample_interval = 2;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would use int32 inline

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since it will be also used in Metadata I decided to create its own message to keep the same semantic. What do you think?

int32 max_interval_ms = 1;
}

enum FilterError {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this shall be aligned with grpc errors? because we encountered difficulties with own error codes and stuff like that right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The purpose of this error enum was to transmit Provider's context errors, like can network error, provider overloaded, etc

@@ -44,6 +45,7 @@ service VAL {
// PERMISSION_DENIED if access is denied for any of the requested signals.
// INVALID_ARGUMENT if the request is empty or provided path is too long
// - MAX_REQUEST_PATH_LENGTH: usize = 1000;
// UNAVAILABLE if there is no provider currently providing the signals
//
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using UNAVAILABBLE for this purpose has some drawbacks:

  1. Clients cannot easily distinguish, if just the signal has no provider or if the whole data broker is unavailable
  2. I would strongly prefer to get the state of the other available signals even if some of them are "not provided" at the moment

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decided after meeting to remove it and keep the non availability of provider as None in datapoint

@@ -298,6 +309,28 @@ message BatchActuateStreamResponse {
Error error = 2;
}

message UpdateFilterRequest {
// Databroker sends filters to provider.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A request id isn't required here?
Can we make sure, that the data broker send only one UpdateFilterRequest to a provider and waits for the response before sending another UpdateFilterRequest?

FILTER_ERROR_CODE_OK = 1;
FILTER_ERROR_CODE_NETWORK_ERROR = 2;
FILTER_ERROR_CODE_OVERLOAD = 3;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't there be more errors like

  • Unknown signal id

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WIll added it

@@ -47,6 +47,24 @@ message Value {
}
}

message SampleInterval {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why SampleInterval? Why not SampleRate, or maybe Frequency, maybe as mentioned in some of the other findings it can be inlined because it's just an object with a single property.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is used in Metadata and Filter messages, wanted to keep same semantic and meaning between both messages.

@@ -131,6 +131,7 @@ service VAL {
// NOT_FOUND if the specified root branch does not exist.
// UNAUTHENTICATED if no credentials provided or credentials has expired
// INVALID_ARGUMENT if the provided path or wildcard is wrong.
// UNAVAILABLE if there is no provider currently providing the signal -> do we want this error for list metadata?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't do this: This prevents clients - including providers - from getting the metadata of a signal if a provider for it is not (yet) available.

Comment on lines -271 to -273
message PublishValueResponse {
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops?

map<int32, Datapoint> data_points = 2;
}

message PublishValuesResponse {
int32 request_id = 1;
uint64 request_id = 1; /// Unique request id for the stream that can be used to match the corresponding request.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missed that yesterday: Do we really need a 64 bit request id? This would mean, we expect that request out of 4 billion could potentially overlap ...
BTW: Wouldn't that be an incompatible interface change - requiring to proceed to v3?

@@ -131,6 +131,7 @@ service VAL {
// NOT_FOUND if the specified root branch does not exist.
// UNAUTHENTICATED if no credentials provided or credentials has expired
// INVALID_ARGUMENT if the provided path or wildcard is wrong.
// UNAVAILABLE if there is no provider currently providing the signal -> do we want this error for list metadata?
//
rpc ListMetadata(ListMetadataRequest) returns (ListMetadataResponse);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add an error to PublishValue for the case that the addressed signal is already "claimed" by another provider (via OpenProviderStream -> ProvideSignalRequest). E.g.
// RESOURCE_EXHAUSTED if the addressed signal is already occupied by another provider
Or what is the current behavior of the broker in those cases?

Comment on lines 288 to +297
message ProvideActuationResponse {
}

message ProvideSignalRequest {
map<int32, SampleInterval> signals_sample_intervals = 1;
}

message ProvideSignalResponse {
}

Copy link
Contributor

@BjoernAtBosch BjoernAtBosch Feb 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I didn't recognize till now that the ProvideSignalRequest/Response is also new.

We should specify the behavior when a provider freshly claims a signal: Does he immediately start to send updates or does he wait until a UpdateFilterRequest comes in? Or should we send back a filter map in the response (like in the UpdateFilterRequest) which is defining the initial filter? What about "legacy" providers not knowing the filter concept?

And BTW: We did not specify the behavior if already another provider has claimed signals for actuation or provisioning. Maybe there should be some respective "error signalling" in the responses.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants