New response format (June 2023) #69
Replies: 6 comments 12 replies
-
|
👋 First off, thank you for exploring the topic of streaming response in GraphQL and pushing the proposal forward over the past several years. At Shopify, we’ve recently begun exploring I see significant benefits to the new design, but also some space for possible refinements. I think the most important aspect of the new design is enforcing no duplicate delivery of any fields. This does add some complexity to the backend—in diffing the different branches and finding the overlapping leaves—but that is a cost that only needs to be paid once per query. I know past discussions considered merging fragments together to a single deferred response, so I’m glad to see this proposal allows each deferred fragment to be delivered independently. Where I would like to understand the proposal more is around the Cache keyIn past discussions, it was suggested that the usage of
It isn’t clear to me why State managementIn the proposed design, clients who receive an Improving
|
Beta Was this translation helpful? Give feedback.
-
|
Awesome, happy to hear that you're using this and you have feedback!
Consider this query: {
a {
... @defer {
b
... @defer {
c
}
... @defer {
d
}
}
}Here we have three different defers that each share the path
Client should not attempt to interpret the The server can produce it by following the spec; e.g. in my spec proposal it's just an auto-increment value: graphql/graphql-spec@
Labels are no longer significant, they only have meaning to the client and not the server. Include or omit them at will. If provided, they will be included with the Path/label combinations are therefore not guaranteed nor required to be unique.
Please note that
The data only needs to be maintained whilst an id is still pending, once the entry is completed it can be forgotten about. Even for a client that does not "care" about tracking the loading status of the various
The It was determined that many users will not want to write their own labels (preferring to just use
We've discussed an alternative to this to help clients migrate by implementing a "client network middleware" that can translate the new data format to the old format for older clients. One major advantage of this approach is that the final solution that we come up with will not have to be swayed by earlier non-optimal forms of the solution. I hope this helps! |
Beta Was this translation helpful? Give feedback.
-
|
Want to echo the thanks for the feedback, I think it's very helpful. The utility of the
|
Beta Was this translation helpful? Give feedback.
-
|
Thank you both for your thoughtful replies!
I believe I phrased this poorly. I understand that the ID is opaque, but consumers still need to learn about its semantics (it does not repeat…but is that true over the course of the entire response? Can they repeat over subsequent responses? Clients will need to understand this to know if they can leverage a global ID cache or need a cache per request/response stream).
I appreciate the clarity of this statement as its what I was struggling with at the beginning: why does I do wonder if we are sacrificing ergonomics for the sake of what might amount to a couple of kilobytes spread out over multiple messages though. Is this a discussion that has happened for other parts of the GraphQL spec? How do you evaluate the priority between terseness and usability?
My concern is less about the volume of state that needs to be recorded, but the fact that any state is required in order to interpret the subsequent responses. It would be particularly elegant to be able to hand off streamed responses to a pipeline that can parse, process, and apply each incremental patch without having to maintain some connection to earlier requests beyond the raw data. As well, payloads beyond the initial response can themselves include new
Indeed 😄 Without labels, the client would need to group all matching paths listed in
If I agree that we should not require labels for clients to use the protocol. This was a sentiment that @benjie echoed above as well:
I would like the protocol to be something that is straightforward to adopt for simple clients, but with enough information to allow more advanced clients to optimize as they wish. The current requirement of This is presumably a rare occasion in GraphQL where the client doesn't have the freedom to specify the shape of the response they want since the format is forced for all responses. How radical would it be to have clients opt into the payload-size-optimizing format as an argument to the directive itself? For example: {
a {
... @defer(pathIds: true) { }
}I can see that getting a little bit confusing if you start to nest |
Beta Was this translation helpful? Give feedback.
-
It is my understanding that the
So I think they clients will need a cache per request/response stream.
The introduction of deduplication requires maintenance of state for the duration of a response within the incrementally built "final reconcilable object." This can be done solely by processing the incremental entries if they contain complete paths rather than
We do not merge defers with common paths, but rather send each defer as it completes; it could be that operations include
Argument-controlled differences in payload format could definitely work, but I imagine it would be on a per client level rather than a per defer? |
Beta Was this translation helpful? Give feedback.
-
|
@jahfer thanks again for spending time with the defer/stream proposal and providing feedback. I think @benjie and @yaacovCR covered most of my sentiments, but I just wanted to add a few additional notes. idAs you noted, we think one of the major benefits of the labelIn your examples, you recommend using query {
friends {
pet {
@defer(label: "petNameLabel") {
name
}
}
}
}Assuming the [
{ "id": "0", "path": ["friends", 0, "pet"], "label": "petNameLabel" },
{ "id": "0", "path": ["friends", 1, "pet"], "label": "petNameLabel" },
{ "id": "0", "path": ["friends", 2, "pet"], "label": "petNameLabel" },
]In this case the same label can refer to multiple different base paths. This is why we chose to use an We hope to release an alpha version of graphql-js with this new response format soon. We appreciate this feedback, please let us know if you have any more concerns. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you so much Rob!! What a milestone to celebrate! Can we revisit the decision of throwing away the entire defer fragment if it sets a field that's delivered previously? An important principle we should consider always follow is that a query executed with @defer/@stream should always deliver the same amount of data as the same query without @defer/@stream. AKA the amount of the information should be the same, despite the format difference. In your example below, if we executed the query without @defer, we would have delivered "anotherField" when an error occurs on "qux". However, when we apply @defer to the query, we no longer have "anotherField".
|
Beta Was this translation helpful? Give feedback.
-
|
@Keweiqu happy to revisit this, I agree it's not ideal. We made this decision because we wanted to ensure that
I also posted this in FAQs in the parent post:
I am open to alternate ways to handle this case, do you have a recommendation? |
Beta Was this translation helpful? Give feedback.
-
|
It's also worth noting that this issue won't apply if null bubbling is disabled (a topic under discussion by the nullability WG). |
Beta Was this translation helpful? Give feedback.
-
Consider an altered version of query ExampleCannotBubbleIntoTheOriginalResult {
me {
foo {
bar {
baz {
... @defer(label: "HereComesQux") {
qux
}
}
}
}
}
}In this case, were a server not to implement defer, the null at qux would bubble to baz, but a server that supports defer would send baz, and no change to the response format is possible that would fix that. Considering we cannot absolutely strictly adhere to the principle articulated above no matter what our other constraints are with respect to incremental delivery, I think it is reasonable to introduce another discrepancy in order to support some of the features/constraints above, i.e. no duplication and consistent delivery of fragments (which lead to the discrepancy in Example H). Separately, @robrichard, I think there is an error in Example H, or at least a divergence from the implementation. If Delivery Group B were not to fail, it would partially overlap with Delivery Group A, and so I think the response would be more like in Example D: [
{
"data": {
"me": {}
},
"pending": [
{"id": "0", "path": [], "label": "A"},
{"id": "1", "path": ["me"], "label": "B"}
],
"hasNext": true
},
{
"incremental": [
{
"id": "0",
"subPath": ["me"],
"data": {
"foo": {
"bar": {}
}
}
},
{
"id": "0",
"subPath": ["me", "foo", "bar"],
"data": {
"baz": "BAZ"
}
}
],
"completed": [
{"id": "0"}
],
"hasNext": true
},
{
"completed": [
{
"id": "1",
"errors": [
{
"message": "Cannot return null for non-nullable field Bar.qux.",
"locations": [...],
"path": ["foo", "bar", "qux"]
}
]
}
],
"hasNext": false
}
]@Keweiqu using this view of the response, I think it's easier for me to reason about why a potential solution for the discrepancy you note is so difficult: Consider what would happen if we updated the response to send [
{
"data": {
"me": {}
},
"pending": [
{"id": "0", "path": [], "label": "A"},
{"id": "1", "path": ["me"], "label": "B"}
],
"hasNext": true
},
{
"incremental": [
{
"id": "0",
"subPath": ["me"],
"data": {
"foo": {
"bar": {}
}
}
},
{
"id": "0",
"subPath": ["me", "foo", "bar"],
"data": {
"baz": "BAZ"
}
}
],
"completed": [
{"id": "0"}
],
"hasNext": true
},
{
"incremental": [
{
"id": "1",
"data": {
"anotherField": "ANOTHER_FIELD"
}
],
"completed": [
{
"id": "1",
"errors": [
{
"message": "Cannot return null for non-nullable field Bar.qux.",
"locations": [...],
"path": ["foo", "bar", "qux"]
}
]
}
],
"hasNext": false
}
]From the perspective of delivery group B, the execution group for the object |
Beta Was this translation helpful? Give feedback.
-
|
@yaacovCR thanks, I fixed Example H above. |
Beta Was this translation helpful? Give feedback.
-
|
Hey @robrichard, In urql we've implemented all three spec variations that have been part of |
Beta Was this translation helpful? Give feedback.
-
|
A few typos in Example D I think it should be: [
{
"data": { "me": {} },
"pending": [
{ "path": [], "id": "0" },
{ "path": ["me"], "id": "1" }
],
"hasNext": true
},
{
"incremental": [
{
"id": "1",
"data": { "list": [{ "item": {} }, { "item": {} }, { "item": {} }] }
},
{ "id": "1", "subPath": ["list", 0, "item"], "data": { "id": "1" } },
{ "id": "1", "subPath": ["list", 1, "item"], "data": { "id": "2" } },
{ "id": "1", "subPath": ["list", 2, "item"], "data": { "id": "3" } }
],
"completed": [{ "id": "1" }],
"hasNext": true
},
{
"incremental": [
{ "id": "0", "subPath": ["me", "list", 0, "item"], "data": { "value": "Foo" } },
{ "id": "0", "subPath": ["me", "list", 1, "item"], "data": { "value": "Bar" } },
{ "id": "0", "subPath": ["me", "list", 2, "item"], "data": { "value": "Baz" } }
],
"completed": [{ "id": "0" }],
"hasNext": false
}
] |
Beta Was this translation helpful? Give feedback.
-
|
Updated, thank you! |
Beta Was this translation helpful? Give feedback.
-
|
Example F is problematic for Apollo Kotlin. We need a way for the generated parser to know it can read the fields of the first fragment. In other examples it looked like For more context, here’s how the generated models will look like: class ExampleF {
class Data {
val me: Me
class Me {
val onUser: OnUser?
}
class OnUser {
val onUser: OnUser1?
}
class OnUser1 {
val a: String?
val b: String?
}
}
}Here’s a version that would work for us: [
{
"data": {
"me": {}
},
"pending": [
{"id": "0", "path": ["me"], "label": "A"}
{"id": "1", "path": ["me"], "label": "B"}
],
"hasNext": true
},
{
"incremental": [
{"id": "1" , "data": {"a": "A", "b": "B"}}
],
"completed": [
{"id": "0"},
{"id": "1"},
],
"hasNext": false
}
] |
Beta Was this translation helpful? Give feedback.
-
|
When you get the first payload: Since there is no pending for label A, can you assume that the fragment has been inlined and has already be returned? |
Beta Was this translation helpful? Give feedback.
-
|
So instead of tracking what's completed and only read completed fragments, we'll need to track what's pending, and skip reading the pending ones. This should be doable - I'll report back. |
Beta Was this translation helpful? Give feedback.
-
|
On Apollo Kotlin's side, this version looks good. Changes to go from the 2022 version we currently implement to this one can be seen here. |
Beta Was this translation helpful? Give feedback.
-
Over the past few months the incremental delivery working group has been working on the issues of data duplication, response amplification, and data consistency. The result of this is an updated response format for the
@defer&@streamproposal, which contains some significant differences from the previous iterations. We would now like to get wider feedback on this proposal.Features
The
incrementalarray is the actual data to be applied to the response, while thependingandcompletedarrays return "metadata" about the execution. They are used to inform clients that defers are being executed and when all fields for that defer have been delivered.To ensure consistency, clients are expected to process all objects in the
incrementalarray for a given payload before re-rendering the associated UIs.Defer Examples
Example A
Overlapping fields from initial payload are not sent in subsequent payloads. Fragment consistency is preserved. Even if "MyFragment" is ready earlier, it is not sent until "j" is also ready.
Example Result:
Example A2
Overlapping fields from parent defers are also not duplicated
Example Result:
Example B
Overlapping fields e & f are sent with whichever fragment completes first, and not sent in subsequent payloads.
{ a { b { c { d } ... @defer(label: "Red") { e { f } potentiallySlowFieldA } } } ... @defer(label: "Blue") { a { b { e { f } } } g { h } potentiallySlowFieldB } }Example Result when
potentiallySlowFieldAcompletes first:Example Result when
potentiallySlowFieldBcompletes first:Example D
Non-overlapping fields in child selections of overlapping fields are sent in separate incremental objects.
Example Result
Example F
Defer fragments with no fields are skipped entirely
Example Result
Example G
Deferred fragments at the same path are delivered independently
Example Result
Example H
If a field in a subsequent defer nulls a previously sent field due to null bubbling, the entire fragment will not be delivered. Clients should treat this fragment similar to a fragment that is
@skip(if: true).Assume
bazresolves beforequxand qux is non-nullable and returns null.Example Result
Stream Examples
Example I
Example J
If after some list fields are streamed:
A "completed" object is sent for the stream with the errors and no more streamed results will be sent
Solution Criteria Evaluation
FAQ's
Why not
done: trueonincrementalinstead ofcompleted?It's possible for a single incremental object result to "complete" more than one deferred fragment. In this example assume
baztakes longer to resolve than the other fields.{ me { ... @defer(label: "A") { foo bar } ... @defer(label: "B") { bar baz } ... @defer(label: "C") { foo baz } } }Example Response
Should
incremental[i].idbe an array, since data is shared by multipleids?No, because then it would introduce confusion on which id is the prefix of the subpath. This single id does not signify that this is the only fragment being delivered as multiple fragments can be delivered together. We will prefer the
idwith the longestpathinpendingto minimize the size of the subpath.Regarding example H, I'm not sure why "anotherField" need to suffer the casualty and not be delivered?
If we delivered the other fields in the fragments it could put clients into a bad state where they understand that all the fields from defer "B" have been delivered, but it has an object for "bar" and no result for "qux". Since the schema has a non-null constraint on "qux" this is a state that should be impossible.
We discussed options where
baris resent as null. Maybe there's some advanced clients that can simultaneously render both states if they occur in unrelated UI components, but if you are constructing the final reconciled object you end up either dropping the whole fragment, rendering the fragment in the above described invalid state, or remove data that has already been displayed to the user.This should only happen when non-null fields are shared across sibling defers. We were thinking this situation could potentially be addressed in the future by the client controlled nullability proposal.
Beta Was this translation helpful? Give feedback.
All reactions