-
Notifications
You must be signed in to change notification settings - Fork 114
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Chat application + labels #515
Conversation
dc38293
to
7936f79
Compare
5e1ae56
to
091aafc
Compare
7af274d
to
7cdcc26
Compare
This is ready for a first round of review, even though the labels only exist in the SDK and not in the actual runtime. I would rather not implement labels in the current C++ runtime just to throw them away and re-implement them in Rust when #540 is done. Note to reviewers: please let me know if you have any suggestions on how to improve the API around labels in particular. |
examples/chat/module/rust/src/lib.rs
Outdated
.rooms | ||
.entry(grpc_invocation_label.clone()) | ||
.or_insert_with(|| create_room_node(&grpc_invocation_label)); | ||
room_node_sender.send(&grpc_invocation) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This feels like it provides a very specific solution that moves the problem elsewhere (forwarding the gRPC channel to another room) rather than providing a generic mechanism that can be used for other routing use cases where the incoming channel into the node is not gRPC.
I assume the more generic solution will come later with a more complete label system implementation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This node solves the problem of having a single inbound channel, over which additional channel handles are received, but these channels have more restrictive labels attached to them. So in this sense, a generic router node would still need to know how to unpack incoming channel handles and inspect their labels, and I am not sure how to abstract that. Perhaps add a Routable
trait in the SDK, implement it for Invocation
, and then make this node generic over that? I'm adding a TODO for now, but let me know if you were thinking of a different problem, or had a different solution in mind.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The more generic problem I was thinking of is routing things other than channels, such as individual messages. The current solution means that every time routing needs to change for a new message, a client needs to make a new connection. For the chat room that means creating a separate gRPC connection for every chat room in which you would like to participate. While this is OK for a small-scale chat app, this does not seems scalable for many potential applications.
But this is outside of the scope of this PR, as it is dependent on future IDL work to support more fine-grained labelling.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fair point, I think this would be addressed (or at least facilitated) by making it easier for client to specify labels for each call, as opposed to only once per channel. WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, per-call labels should help to resolve this.
examples/chat/module/rust/src/lib.rs
Outdated
} | ||
}; | ||
fn handle_grpc_invocation(&mut self, grpc_invocation: Invocation) -> Result<(), oak::OakError> { | ||
let grpc_invocation_label = grpc_invocation.request_receiver.label()?; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The label is now considered public, since the node is not getting tainted by reading the label.
This means in principle that anyone might get hold of the label. I know this is just a toy example, but at the very least the validity of the label should be checked somewhere (eg that the label can be generated from the bearer token). The label is used for routing to a room, which would mean anyone who has the public label could send messages to the room (or subscribe to it) without knowing the bearer token. A malicious client could manually override the gRPC metadata to use the label.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is WAI for now. The label is the equivalent of a public key, and in principle, anyone who has a public key can encrypt data using it and send it to some storage service somewhere, so that the owner of the corresponding private key can decrypt it. This only guarantees secrecy, but no provenance. If we wanted a room to which only some people could send data, then we need to involve integrity labels, which as you said, will be based on knowing the actual bearer token. Clarified the comment above.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed. The fact that someone can subscribe to a room using the public key still seems potentially problematic. They will be blocked from reading the contents once the label declassification system is implemented, but the ability to subscribe to content that you cannot access is a potential source of confustion.
sdk/rust/oak/src/io/receiver.rs
Outdated
@@ -66,6 +66,11 @@ impl<T: Decodable> Receiver<T> { | |||
T::decode(&message) | |||
} | |||
|
|||
#[allow(clippy::trivially_copy_pass_by_ref)] | |||
pub fn label(&self) -> Result<crate::label::Label, OakError> { | |||
Ok(crate::label::Label(vec![1, 2, 3])) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Worth having a TODO(#NNN): text
marker? (Although I guess it will be pretty obvious where the plumbing to the ABI should go).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added TODO. I will create an issue and attach to it later in the review process if we decide to stick with this API.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added issue # now too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Issue number? (passim for TODOs)
sdk/rust/oak/src/label.rs
Outdated
} | ||
|
||
pub fn get_node_label() -> Result<Label, crate::OakStatus> { | ||
// TODO: Implement this via `oak_abi`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there an issue number we could put here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As for the other TODO, I am postponing creating issues until we generally agree on the shape of the API.
examples/chat/proto/chat.proto
Outdated
@@ -64,5 +49,5 @@ service Chat { | |||
rpc Subscribe(SubscribeRequest) returns (stream Message); | |||
|
|||
// Send a message to a chat room. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This proto definition is now a bit incomplete, because some of the information relevant to the API is now carried separately from the .proto message definitions (the "room ID" is effectively in metadata not data).
That obviously correlates with the need for the Router node to be restricted to looking at metadata but not data, but having a incomplete/split API definition seems awkward.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes that's true, and I don't have a good answer to that. The best I can think of is that this is similar to having a JSON schema for a web API, but that would not capture things like URLs or headers, so you would have to carry that context around in some other way. I think there are ways of bridging that gap, for instance by having web frameworks that convert URL fragments to JSON fields in a request, but I am not sure what would work best here.
At a more philosophical level though, the point of Oak is to minimize the trust boundary, so that a client device would only need to convince the user that some labels are applied to any outgoing message, regardless of what the message actually contains, so the distinction between trusted metadata and untrusted data is actually quite useful. For instance, a chat app Android client would only need to show that it is attaching some labels to outgoing gRPC requests, and that should be all the user cares about in terms of information flow.
That said, I'd be curious to hear others point of view on this.
/cc @project-oak/core
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree with the philosophical point above about what a client device cares about, but the model is not great from a developer point of view. The developer now has to think about every API in terms of this split view. Validating consistency between data and metadata is more difficult as these are not tightly coupled. This is further complicated by the metadata being defined per connection and the data per message.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have added some more comments here.
Conceptually this is similar to how in an API you may have methods that return data based on the provided authentication credentials, for instance all email messages for the authenticated user, etc. Though here the relationship is complicated by the fact that the label of the request is used to select the chat room to access. Perhaps we should go back to using some other identifier for chat rooms instead of the label itself, but then it would not be possible to route based on it because the room identifier would be protected by the label and not accessible by the router node. I think this may be solvable by an additional layer of routing nodes, but I am not sure that's the best way to go, so I'd leave that experiment and discussion for a future PR.
I also added a new method GetAllMessagesCount
, which is supposed to return the number of messages from all chats that the user has access to. It does not seem practical (possible?) for it to be implemented in the current system, so I left it unimplemented, but I think it is a good discussion point to consider for future label-related discussions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Update: I removed GetAllMessagesCount
again, I'll leave that for another time, since currently it does not really add much value.
docs/programming-oak.md
Outdated
sender: oak::io::Sender::new(wh), | ||
admin_token, | ||
} | ||
let (wh, rh) = oak::channel_create(grpc_invocation_label).unwrap(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This leaves the doc with a bit of a puzzle in it (what's the mysterious grpc_invocation_label
?) so maybe leave a TODO(#NNN)
to sort out?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Clarified a bit in the previous paragraph now.
examples/chat/module/rust/src/lib.rs
Outdated
use protobuf::Message; | ||
use std::collections::hash_map::Entry; | ||
use oak::grpc::Invocation; | ||
use protobuf::ProtobufEnum; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can now drop oak_derive
from [dependencies]
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's still needed by backend.rs
.
examples/chat/module/rust/src/lib.rs
Outdated
impl OakNode for Node { | ||
fn new() -> Self { | ||
#[no_mangle] | ||
pub extern "C" fn router_oak_main(handle: u64) -> i32 { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If this is the main entrypoint for this Node, then I think it should be -> ()
(since cb8da90), and there probably needs to be a change somewhere in the client to look for "router_oak_main"
rather than the default "oak_main"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done. Will probably have to move things around when #462 is closed, I am happy to wait until that's done before submitting this, so we don't have to fix it twice.
ef3e04e
to
b47d2c9
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the comments so far
examples/chat/module/rust/src/lib.rs
Outdated
use protobuf::Message; | ||
use std::collections::hash_map::Entry; | ||
use oak::grpc::Invocation; | ||
use protobuf::ProtobufEnum; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's still needed by backend.rs
.
examples/chat/module/rust/src/lib.rs
Outdated
impl OakNode for Node { | ||
fn new() -> Self { | ||
#[no_mangle] | ||
pub extern "C" fn router_oak_main(handle: u64) -> i32 { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done. Will probably have to move things around when #462 is closed, I am happy to wait until that's done before submitting this, so we don't have to fix it twice.
examples/chat/module/rust/src/lib.rs
Outdated
} | ||
}; | ||
fn handle_grpc_invocation(&mut self, grpc_invocation: Invocation) -> Result<(), oak::OakError> { | ||
let grpc_invocation_label = grpc_invocation.request_receiver.label()?; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is WAI for now. The label is the equivalent of a public key, and in principle, anyone who has a public key can encrypt data using it and send it to some storage service somewhere, so that the owner of the corresponding private key can decrypt it. This only guarantees secrecy, but no provenance. If we wanted a room to which only some people could send data, then we need to involve integrity labels, which as you said, will be based on knowing the actual bearer token. Clarified the comment above.
examples/chat/module/rust/src/lib.rs
Outdated
.rooms | ||
.entry(grpc_invocation_label.clone()) | ||
.or_insert_with(|| create_room_node(&grpc_invocation_label)); | ||
room_node_sender.send(&grpc_invocation) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This node solves the problem of having a single inbound channel, over which additional channel handles are received, but these channels have more restrictive labels attached to them. So in this sense, a generic router node would still need to know how to unpack incoming channel handles and inspect their labels, and I am not sure how to abstract that. Perhaps add a Routable
trait in the SDK, implement it for Invocation
, and then make this node generic over that? I'm adding a TODO for now, but let me know if you were thinking of a different problem, or had a different solution in mind.
sdk/rust/oak/src/label.rs
Outdated
} | ||
|
||
pub fn get_node_label() -> Result<Label, crate::OakStatus> { | ||
// TODO: Implement this via `oak_abi`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As for the other TODO, I am postponing creating issues until we generally agree on the shape of the API.
docs/programming-oak.md
Outdated
sender: oak::io::Sender::new(wh), | ||
admin_token, | ||
} | ||
let (wh, rh) = oak::channel_create(grpc_invocation_label).unwrap(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Clarified a bit in the previous paragraph now.
examples/chat/client/chat.cc
Outdated
@@ -185,40 +179,37 @@ class OakApplication { | |||
// RAII class to handle creation/destruction of a chat room. | |||
class Room { | |||
public: | |||
// Caller should ensure stub outlives this object. | |||
// Caller must ensure stub outlives this object. | |||
Room(Chat::Stub* stub) : stub_(stub) { | |||
oak::NonceGenerator<64> generator; | |||
grpc::ClientContext context; | |||
CreateRoomRequest req; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: This seems to be unused, unless I am missing something.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
examples/chat/client/chat.cc
Outdated
auto admin_token = generator.NextNonce(); | ||
req_.set_admin_token(std::string(admin_token.begin(), admin_token.end())); | ||
admin_token_ = oak::NonceToBytes(generator.NextNonce()); | ||
CreateRoomRequest req_; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: As this is now a local variable and no longer a private field, it should probably not end with _
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
examples/chat/module/rust/src/lib.rs
Outdated
} | ||
}; | ||
fn handle_grpc_invocation(&mut self, grpc_invocation: Invocation) -> Result<(), oak::OakError> { | ||
let grpc_invocation_label = grpc_invocation.request_receiver.label()?; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed. The fact that someone can subscribe to a room using the public key still seems potentially problematic. They will be blocked from reading the contents once the label declassification system is implemented, but the ability to subscribe to content that you cannot access is a potential source of confustion.
examples/chat/module/rust/src/lib.rs
Outdated
.rooms | ||
.entry(grpc_invocation_label.clone()) | ||
.or_insert_with(|| create_room_node(&grpc_invocation_label)); | ||
room_node_sender.send(&grpc_invocation) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The more generic problem I was thinking of is routing things other than channels, such as individual messages. The current solution means that every time routing needs to change for a new message, a client needs to make a new connection. For the chat room that means creating a separate gRPC connection for every chat room in which you would like to participate. While this is OK for a small-scale chat app, this does not seems scalable for many potential applications.
But this is outside of the scope of this PR, as it is dependent on future IDL work to support more fine-grained labelling.
examples/chat/proto/chat.proto
Outdated
@@ -64,5 +49,5 @@ service Chat { | |||
rpc Subscribe(SubscribeRequest) returns (stream Message); | |||
|
|||
// Send a message to a chat room. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree with the philosophical point above about what a client device cares about, but the model is not great from a developer point of view. The developer now has to think about every API in terms of this split view. Validating consistency between data and metadata is more difficult as these are not tightly coupled. This is further complicated by the metadata being defined per connection and the data per message.
114b3eb
to
ba81e25
Compare
73bd770
to
d09cd34
Compare
@anghelcovici @daviddrysdale @conradgrobler could you take another look? Especially around the proposed SDK changes (creating channels and nodes with labels, etc.) I also updated the documentation for the chat app, let me know if you think it's reasonable. |
docs/programming-oak.md
Outdated
@@ -382,19 +382,18 @@ pattern for the existing Node is to: | |||
needed. | |||
|
|||
For example, the [example Chat application](../examples/chat) creates a Node for | |||
each chat room and saves off the write handle that will be used to send messages | |||
to the room: | |||
each chat room (using the label corresponding to the incoming gRPC invocation) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Be nice if label
was [label](concepts.md#labels)
at some point …
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Started on this now, PTAL.
examples/chat/README.md
Outdated
- The **admin ID** allows destruction of the room instance at the server; the | ||
client keeps hold of this ID internally. | ||
Additionally, an **admin token** allows destruction of the room instance at the | ||
server; the client keeps hold of this ID internally. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nits: be nice to match up "token" and "ID", and to mention where the token comes from, e.g.:
"When creating a room, the client also includes an admin token in the room creation request. The client keeps hold of this token and can use it to destroy the room instance at the server."
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
examples/chat/README.md
Outdated
|
||
This will emit a trace line that holds the information needed to: | ||
The application starts with a "router" node that only ever handles public data, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: I've generally tried to capitalize Application and Node.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
examples/chat/README.md
Outdated
This will emit a trace line that holds the information needed to: | ||
The application starts with a "router" node that only ever handles public data, | ||
and is not allowed to inspect any data protected by a label (e.g. the | ||
aforementioned authentication bearer token). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: not completely clear whether the parenthetical content maps to the "data" or the "label" (that protects that data).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reworded.
examples/chat/README.md
Outdated
A client can then create a **room**; which returns a pair of random IDs that act | ||
as bearer tokens: | ||
A client can then create a **room**, which is protected by an authentication | ||
bearer token that must be shared with any other participant to be invited to the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe mention "label" or "metadata" here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
Ok(Empty::new()) | ||
} | ||
|
||
fn destroy_room(&mut self, req: DestroyRoomRequest) -> grpc::Result<Empty> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do room Nodes terminate, or are destroyed rooms just going to pile up?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good question: according to the labelling rules, the Router node should not be able to learn that a room was destroyed (because such information is not public). But it feels like we should at least allow termination based on private data, otherwise no non-public node would ever be able to terminate. Not sure how / if the Router node would be able to notice though.
Created #607 to track.
examples/chat/module/rust/src/lib.rs
Outdated
/// from "public". | ||
#[derive(Default)] | ||
struct Router { | ||
rooms: HashMap<oak::label::Label, oak::io::Sender<Invocation>>, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How would an entry ever get dropped from Router.rooms
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It cannot at the moment, added a comment to clarify.
examples/chat/module/rust/src/lib.rs
Outdated
.entry(grpc_request_label.clone()) | ||
.or_insert_with(|| create_room_node(&grpc_request_label)); | ||
// Forward the incoming gRPC invocation to the appropriate room node. | ||
room_node_sender.send(&grpc_invocation) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Once the Invocation
has been forwarded, should the local copy be closed (i.e. both of the underlying handles closed)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes but closing all these channels is getting tedious. Since they are now almost always created using higher level SDK functions, we should consider a more RAII-oriented approach (i.e. close them on drop). WDYT?
examples/chat/module/rust/src/lib.rs
Outdated
} | ||
fn create_room_node(grpc_invocation_label: &oak::label::Label) -> oak::io::Sender<Invocation> { | ||
let (wh, rh) = oak::channel_create_with_label(grpc_invocation_label).unwrap(); | ||
oak::node_create_with_label("app", "oak_main", rh, grpc_invocation_label) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are labels intended to be static (specified up-front at channel/node creation time) or dynamic (extended when labelled data arrives) or both?
I guess I'd imagined more dynamic behaviour, which seems like it would make the explicit specification of labels here unnecessary – the new per-Room Node would start with an empty label, but would acquire the room label on its first read of a request message.
(Aside: it's also quite tricky to keep track of the different levels of indirection: the channel that the Invocation
arrives on has no labels, but the Invocation
itself contains two channels which do have the labels, and this Invocation
is forwarded down another channel – which again can stay label-free because the labelled data is in an inner channel. Not sure what to do about that, though, other than having clear docs.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am trying to avoid "floating" labels that are automatically added when reading from channels, since that may introduce covert channels.
That said, it would be possible to implement this (and in fact it was one of my earlier attempts) so that a room node is created with "public" label, and then explicitly changes its label based on the first message it receives. I was not a big fan of it though, because I could not find a corresponding mechanism for channels, so I still had to let the channel creator specify the label for the channel upfront, and that should definitely not change once set. Anyways, this is not set in stone yet, happy to revisit and try out different models if this is confusing.
A more drastic approach could be to get rid of channels entirely, and give handles to nodes, effectively melding channels and nodes together, but that may have other side-effects (e.g. client streaming, as you mentioned a while ago).
examples/chat/module/rust/src/lib.rs
Outdated
} | ||
} | ||
fn create_room_node(grpc_invocation_label: &oak::label::Label) -> oak::io::Sender<Invocation> { | ||
let (wh, rh) = oak::channel_create_with_label(grpc_invocation_label).unwrap(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this channel actually need the grpc_invocation_label
? If only Invocation
s (i.e. pairs of handles) are sent down it, then possibly not?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You are correct, changed back to plain channel_create
.
sdk/rust/oak/src/lib.rs
Outdated
/// Only nodes with a compatible label will be able to read and write to this channel: | ||
/// | ||
/// - A node with label `L_w` may write to a channel with label `L_c` iff `L_w ⊑ L_c`. | ||
/// - A node with label `L_r` may read from a channel with label `L_c` iff `L_c ⊑ L_r`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not currently actionable, but just an obserbation to keep in mind for future design and planning: As they stand, the two rules in combination imply that bidirectional communication is only supported when both nodes have exactly the same labels. I assume these statements will change once we have implemented declassification.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The statement is still correct even in the case of declassification: declassification would allow a node to explicitly "lower" its label, if allowed by the rules, so that it matches that of the other side, at which point they would be able to have bi-directional communication. There are no other exceptions made via declassification, it's only about controlling the node own label.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In a future scenario where nodes are multi-threaded this could be tricky. One thread might try to raise the node's label to read a new message while another might lower the same node's label to write a declassified message. This would potentially require locks spanning large sections of code, effectively making it single-threaded again. A mechanism where a node could declassify a single message without changing its overall label seems easier to use.
examples/chat/module/rust/src/lib.rs
Outdated
let mut grpc_response_writer = | ||
oak::grpc::ChannelResponseWriter::new(grpc_invocation.response_sender); | ||
grpc_response_writer.close(Err(oak::grpc::build_status( | ||
oak::grpc::Code::PERMISSION_DENIED, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this propagated to the client? In that case, it is possible to leak data. As a general rule, all failures should be silent unless in debug mode.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If there was a rule that needs to be manually enforced in the code in order to prevent data leak, then it would be a failure in the labelling system, right? :)
The resulting error is sent over the gRPC response channel, and respects the label on that channel; there is no special treatment for errors (intentionally).
In particular, the error will never reach the caller if it comes from a Node with a label that does not flow to the caller's label. But in this case, the error comes from a public node, and public label flows to any other label.
examples/chat/README.md
Outdated
``` | ||
In a more realistic scenario, the first client would securely share the bearer | ||
token with other participants that it wants to be able to join the chat room, | ||
using some out-of-band mechanism. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: The existence of a secure out of band mechanism for exchanging bearer tokens means that they probably don't need a chat room to exchange information preivately, so this does not really seem like a more realistic scenario. Perhaps add a comment that this will be improved when public-key options are implemented?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think this is true. Even assuming labels representing public keys and more advanced cryptography, Oak does not (and should not, IMO) solve the key distribution problem. It only allows parties that already have each other's public keys to obtain similar guarantees as e2e encryption, but allowing the application to perform operations on the data, instead of only seeing encrypted binary blobs. Does it make sense, or are you thinking that Oak could help at a more fundamental level?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I also don't think that we should solve the key distribution problem. The main difference is between exchanging public keys versus private or symmetric keys. For public keys it is just a matter of integrity, not secrecy, so you can use a registry of public keys that you both trust. In the current implementation sharing the bearer token is equivalent to sharing a private or symmetric key. If I already had a secure out of band mechanism to share such sensitive information that provided the needed secrecy I would just send the message itself through this out of band mechanism.
08ccd50
to
8f97f6b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for all the comments, apologies if I missed any, if you notice I did please reply so they are brought back to my attention.
examples/chat/README.md
Outdated
|
||
This will emit a trace line that holds the information needed to: | ||
The application starts with a "router" node that only ever handles public data, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
examples/chat/README.md
Outdated
|
||
- connect to the same Oak application (with `--app_address`) | ||
- join the chat room (with `--room_id`). | ||
The router node receives gRPC invocations on an inbound chananel, but only looks |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
examples/chat/README.md
Outdated
at the associated label (which is public), and based on the label only, routes | ||
the request to either an existing or a newly created "room worker node", which | ||
is created with that specific label and only ever handles data with the same | ||
label, for the duration of its own lifetime. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
)); | ||
} | ||
|
||
info!("destroying room"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
Ok(Empty::new()) | ||
} | ||
|
||
fn destroy_room(&mut self, req: DestroyRoomRequest) -> grpc::Result<Empty> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good question: according to the labelling rules, the Router node should not be able to learn that a room was destroyed (because such information is not public). But it feels like we should at least allow termination based on private data, otherwise no non-public node would ever be able to terminate. Not sure how / if the Router node would be able to notice though.
Created #607 to track.
examples/chat/README.md
Outdated
``` | ||
In a more realistic scenario, the first client would securely share the bearer | ||
token with other participants that it wants to be able to join the chat room, | ||
using some out-of-band mechanism. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think this is true. Even assuming labels representing public keys and more advanced cryptography, Oak does not (and should not, IMO) solve the key distribution problem. It only allows parties that already have each other's public keys to obtain similar guarantees as e2e encryption, but allowing the application to perform operations on the data, instead of only seeing encrypted binary blobs. Does it make sense, or are you thinking that Oak could help at a more fundamental level?
examples/chat/module/rust/src/lib.rs
Outdated
} | ||
fn create_room_node(grpc_invocation_label: &oak::label::Label) -> oak::io::Sender<Invocation> { | ||
let (wh, rh) = oak::channel_create_with_label(grpc_invocation_label).unwrap(); | ||
oak::node_create_with_label("app", "oak_main", rh, grpc_invocation_label) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am trying to avoid "floating" labels that are automatically added when reading from channels, since that may introduce covert channels.
That said, it would be possible to implement this (and in fact it was one of my earlier attempts) so that a room node is created with "public" label, and then explicitly changes its label based on the first message it receives. I was not a big fan of it though, because I could not find a corresponding mechanism for channels, so I still had to let the channel creator specify the label for the channel upfront, and that should definitely not change once set. Anyways, this is not set in stone yet, happy to revisit and try out different models if this is confusing.
A more drastic approach could be to get rid of channels entirely, and give handles to nodes, effectively melding channels and nodes together, but that may have other side-effects (e.g. client streaming, as you mentioned a while ago).
examples/chat/proto/chat.proto
Outdated
rpc Subscribe(SubscribeRequest) returns (stream Message); | ||
|
||
// Send a message to a chat room. | ||
rpc SendMessage(SendMessageRequest) returns (google.protobuf.Empty); | ||
// Send a message to a chat room identified by the provided label metadata. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Clarified.
examples/chat/proto/chat.proto
Outdated
} | ||
|
||
// For all methods in this service, the request label (determined by gRPC | ||
// metadata) must "flow to" the response label (determined by client | ||
// authentication), unless otherwise specified. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Expanded in the concepts.md file and added a link here.
examples/chat/module/rust/src/lib.rs
Outdated
} | ||
} | ||
fn create_room_node(grpc_invocation_label: &oak::label::Label) -> oak::io::Sender<Invocation> { | ||
let (wh, rh) = oak::channel_create_with_label(grpc_invocation_label).unwrap(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You are correct, changed back to plain channel_create
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for all the comments, apologies if I missed any, if you notice I did please reply so they are brought back to my attention.
65322c7
to
f568ad7
Compare
f568ad7
to
f0c5b0e
Compare
Expose methods to read channel label on Receiver and Sender objects. This change removes quite a bit of complexity from the chat app structure, and makes it actually use labels in a meaningful way, although for now only the ones related to bearer token authentication. Ref project-oak#488 project-oak#608
f0c5b0e
to
6e9dc5f
Compare
Add support for labels to Chat example
Ref #488
Checklist
cover any TODOs and/or unfinished work.
construction.