HTTP parameters for specifying context or frame #8
Comments
gkellogg
added
security-tracker
|
We should align this with the work in the W3C DXWG (e.g., w3c/dxwg#261) |
|
Six kinds of +1. In the field this issue comes up for me at least every three months or so! 😁 The potential for DOS is real, but the need is pressing. |
|
Propose to discuss this at TPAC in a joint session with DXWG. |
|
@azaroth42 Is there a DXWG issue to which we can link this? |
|
w3c/dxwg#261 :) |
|
This issue was discussed in a meeting.
View the transcriptRob Sanderson: we can use space separated lists of URIs, so the profile=”” parameter can contain both the compaction, etc. defined URIs… as well as a JSON-LD context or frame URI … however, these URIs might collectively get quite long Adam Soroka: if you specify a URI don’t you want it to be compacted? Rob Sanderson: it could be a frame Gregg Kellogg: or a bomb Benjamin Young: or a cat photo Adam Soroka: do we want to have a default use like it’s assumed it’s a context URI? Ivan Herman: what would this look like in practice? Rob Sanderson: I’ll write it up…one second… Rob Sanderson: ACCEPT: application/ld+json;profile="http://w3.org/ns/anno.jsonld" Rob Sanderson: this would say, “I want this request compacted according to this context” Gregg Kellogg: if that JSON-LD URL returns a context Benjamin Young: don’t dereferences things found in profile params … because proxies Gregg Kellogg: right, security concerns/needs should be expressed Benjamin Young: we used the entire string as the “media type” for Web Annotation … with no expressed intent or idea of dereferencing that context URI Adam Soroka: I’m not too worried about big headers … if you’re going to be this particular about it, then that’s a “cost” you’ll face Rob Sanderson: and the multiple URIs in profile=”” is not a problem? Adam Soroka: no, I think that’s fine Rob Sanderson: if you don’t care if you’re flattened or compacted, then don’t state it and use the context for whichever you’re returning Benjamin Young: is there priority among these strings? … or do they just work as flags? Rob Sanderson: they’re just flags Adam Soroka: do we want q=”” parameters for this? Rob Sanderson: I’d say it’s an error condition to have multiple contexts given Benjamin Young: is the intent to continue–as Web Annotation did–as a single opaque string? … or are we switching on each piece? … mostly it’s a question for me about upgrading existing Web Annotation servers (and the like) … so are they opaque “whole” strings? or are we taking them apart in some way? Rob Sanderson: so, in the case of compaction at least, we could have a default that maps to compaction … which is how profile=”” is used in the cases that we know Gregg Kellogg: are you going to help write these tests? Rob Sanderson: sure. I’ll help write those tests … and the paragraph/dependencies Gregg Kellogg: and at the moment we don’t have complete HTTP header tests…but need those Rob Sanderson: I know someone (bigbluehat) who’s done this already Benjamin Young: we’re continuing the Web Annotation validation/testing tooling work at http://annotator.apache.org/ Gregg Kellogg: so, there are a few HTTP tests now … they were originally in an .htaccess file … but right now it’s left up to the implementations to act as if the headers were expressed in the responses or not … the test suite should ideally run off a W3C mirror that implement these .htaccess Benjamin Young: curious which bits you needed in .htaccess Gregg Kellogg: there’s a remote access manifest in the API Gregg Kellogg: https://github.com/w3c/json-ld-api/blob/master/tests/remote-doc-manifest.jsonld Gregg Kellogg: for example https://github.com/w3c/json-ld-api/blob/master/tests/remote-doc-manifest.jsonld#L34 … this one tests a application/jldTest content type … which in this case should fail … others here deal with redirects … also, another for the Link header https://github.com/w3c/json-ld-api/blob/master/tests/remote-doc-manifest.jsonld#L84 … for all of these we’re analyzing the JSON-LD result … so we probably want similar ones analyzing the HTTP headers themselves Proposed resolution: Add text to Iana Considerations explicitly allowing the request of a context or frame document, plus security consideration on whitelist, plus tests (Rob Sanderson) Rob Sanderson: +1 Gregg Kellogg: +1 Adam Soroka: +1 Benjamin Young: +1 Ivan Herman: +1 Resolution #5: Add text to Iana Considerations explicitly allowing the request of a context or frame document, plus security consideration on whitelist, plus tests |
|
Something to resolve: If the ACCEPT header includes a json-ld media type with a context or frame profile that the server won't support (e.g., black/white list) I presume it would skip that request and go to the next highest-priority media type it supports (possibly application/ld+json without a profile or without a specific context/frame) returning status 415 if none found. This could mean not returning any JSON-LD if profile cannot be satisfied. Alternatively, the server should satisfy the request as best it can without consideration of the profile parameter. |
|
I'm not sure if I'm getting at your point, @gkellogg, but my understanding of HTTP (based on the old RFC and possibly out of date) is that the mediatype alone is what |
|
I think it's correct that the server is supposed to consider the entire value of the https://tools.ietf.org/html/rfc7231#page-39 I think it would be good practice for clients to specify both a profile and also "unprofiled" JSON-LD (if they can actually accept that), e.g. If a client sends |
|
Wow, I got that completely backwards! Thanks for the correction, @Conal-Tuohy. Given that many clients will be expecting to use compacted JSON-LD as JSON, I think a 4xx is by far the best answer here. Non-compacted forms are going to be more surprising and troublesome than helpful to such clients. |
|
Better than status 415 is status 406 "Not Acceptable":
|
|
This issue was discussed in a meeting.
View the transcript4.1. HTTP parameters for specifying context or frameBenjamin Young: #8 Benjamin Young: * syntax - #111 Benjamin Young: * framing - w3c/json-ld-framing#34 Benjamin Young: * api - w3c/json-ld-api#56 (work in progress) Benjamin Young: gkellogg has the lion’s share of the work done on this Gregg Kellogg: the first PR (syntax doc, #111) pretty much ready to go … removed language about quotes for the value of profile params … which is not true in general … only if multiple URIs are used, and added some examples with different types of profile params Ivan Herman: I had a comment on that … content, not editorial … a sentence says that JSON-LD processors may restrict URIs for context and frames … which I read as saying, “If I’m a JSON-LD processor, I can refuse any context except this particular one” … is that really what we mean Gregg Kellogg: well, this could become an attack vector, so we allow servers to restrict the action Gregg Kellogg: framing is potentially expensive computationally … and people have demonstrated attacks via context … which we think we’ve mostly fixed, but obviously we can’t rule it entirely out … going back to the CG, there was this desire to restrict present there Ivan Herman: i have two problems… … this is clearly non-editorial … have we discussed/voted on this restriction? … shouldn’t be in an editorial PR. … I don’t understand the problem this solves Gregg Kellogg: this is part of the request header. Ivan Herman: oh, the request header? (not the response) Gregg Kellogg: this signals the server as to what the client wants … the server must examine the profile param and if it isn’t acceptable, return a 406 … this new language just gives license to the server to do just that if the context is unacceptable … you could also fall-back in various ways Ivan Herman: Okay, I misunderstood what was done, so that’s okay … but still, this is more than editorial Gregg Kellogg: I don’t say that this is editorial … I folded in the previous discussions on this Ivan Herman: sorry for the interrupt, fine with this. I will put in a link to this discussion into the PR … must be documented, now it is … we just want to document that everyone is fine with this Benjamin Young: I’ve got a nit … about the quotes being optional save for multiple URIs Gregg Kellogg: for spaces … and I show examples of this … we don’t mandate that single spaces must be used … it’s just whitespace as defined by the rFC Benjamin Young: okay, thanks Gregg Kellogg: we have also the PR for the API doc Gregg Kellogg: w3c/json-ld-api#56 Gregg Kellogg: there is w-i-p on the API at that PR … framing was a more minor change Gregg Kellogg: w3c/json-ld-framing#34 Gregg Kellogg: the controversial point bigbluehat raised is whether adding a framing URI changes the semantics enough to make it inappropriate … as a profile param … online discussion indicates not … the change requires the profile param … which changes current behavior for people requesting frames … there might b more changes requireed for this for optinality Gregg Kellogg: the idea was that a frame would be represented by a mimetype … but no one implemented that … and now we’ve moved to using a profile param for frame … so the question is, must the profile param appear, or is it optional/advisory … this param is for getting a doc that is a frame … and would appear on the response as well Benjamin Young: yes, we would want that on the response as well Benjamin Young: https://tools.ietf.org/html/rfc6906 Gregg Kellogg: we need more discussion on that PR Adam Soroka: w3c/json-ld-framing#34 Gregg Kellogg: the API PR (w3c/json-ld-api#56 is just tests for this new behavior … I couldn’t see what language to add to the API, otherwise … the API do is concerned with what happens once you’ve requested a doc and are processing the result … we don’t need to specify how to use an Accept header Benjamin Young: this would only be for requesting a single doc in various representation … I’m glad that framed docs aren’t new mediatypes … but it seems unlikely that one document would be all the things at once (frame, context, instance) Gregg Kellogg: but you aren’t requesting a doc, you are requesting a resource. … e.g. for SPARQL users Gregg Kellogg: profile contains one or more of our URIs, as well as URLs to contexts, frames, etc. David Newbury: a real-world example: we have resources we’d like to be able to provide either as schema.org or CIDOC-CRM … same data, difference context/framing Gregg Kellogg: only the non-registered URLS would be treated as dereferencable … similar to when the mimetype is just application/json and you stick the contet in a header Benjamin Young: this is where I get concerned about violating the RFC– the profile param shouldn’t be dereferenced Gregg Kellogg: good point… … my impression was that we could define this behavior, but we do need to review Benjamin Young: in the Web Annotation we avoided triggering new representations via the profile param, only selecting from extant representations Ivan Herman: https://tools.ietf.org/html/rfc6906 Benjamin Young: this is a supernice feature, but is this where it belongs? Ivan Herman: what the RFC (https://tools.ietf.org/html/rfc6906#section-3) does say is: “A profile MUST NOT change the semantics of the resource representation when processed without profile knowledge, so that clients both with and without knowledge of a profiled resource can safely use the same representation.” Benjamin Young: There is also https://tools.ietf.org/html/rfc4288#section-4.3 Ivan Herman: so this may indeed be a problem Gregg Kellogg: the use of a context does not change the semantics of a representation, just the syntax Benjamin Young: it’s the functionality concern … if these are URLs (not merely URIs) they will be looked up and used for computation Ivan Herman: it is an identifier; to quote further from https://tools.ietf.org/html/rfc6906#section-3: “Profiles are identified by URI. However, as is the case with, for example, XML namespace URIs, the URI in this case only serves as an identifier,” Ivan Herman: it is at the beginning of that section Gregg Kellogg: rfc4288#section-4.3 is a different case of profile: that’s the header vs the mimetype Gregg Kellogg: e.g. dc:name vs schema:name … totally different graphs … but for framing, it’s the same graph … and client can reframe or re-compact or whatever … for bigbluehat’s case, where there’s a range of possible contexts… … two ways to interpret that: first, I want the Schema context vs the DC context … or, I want “name” vs “schema:name” where the terms used are different Benjamin Young: in the case of conneg, you could list several contexts … and the sever will compute on your request and select something … that’s not a processing instruction … that’s a great idea, but it should go in a different box Pierre-Antoine Champin: not fully agree with gkellogg about changing strings in the JSON for different contexts … in both cases, the semantics are the same … the condition for selecting different vocabularies is that the semantics of the vocabularies is the same … in both cases, the changes are at the syntax level, just at concrete or abstract syntax Gregg Kellogg: if we look at the test suite, what might be considered? … one, you can take the result, expand it, and compare … or two, turn them into RDF and compare the graphs for isomorphism … (or in framing, a strict subgraph relationship) Ivan Herman: +1 to gregg David Newbury: I’m assuming that I have all the data and am just selecting from it, not doing new processing Ivan Herman: whither shall we wander? … what do we do if profiles are not the correct choice? … our own request header? … is there an extant header we can adopt? Benjamin Young: Profile seems like the wrong vehicle for this signaling, but we cannot discard the use case Proposed resolution: limit profile parameter use to URI’s, but continue to pursue an alternative to the profile media type parameter for client-signaled frames and contexts URLs for servers to (potentially) use when providing responses (Benjamin Young) Gregg Kellogg: https://www.w3.org/TR/json-ld11/#interpreting-json-as-json-ld Gregg Kellogg: we might borrow from the way we handle application/json … in a response header, we add a link header targeting the context … we could use a Link header on the request Ivan Herman: we would need a more detailed description Benjamin Young: https://tools.ietf.org/html/rfc8288 Benjamin Young: you sure can use a Link header on the request … but let’s get to a proposal … we need to get URLs out of the profiles … and there is clear desire in the group for this functionality … link rel= might be a nice way to do that Proposed resolution: limit profile parameter use to URI’s, but continue to pursue an alternative to the profile media type parameter for client-signaled frames and contexts URLs for servers to (potentially) use when providing responses (Benjamin Young) Gregg Kellogg: +1 Benjamin Young: +1 Adam Soroka: +1 Jeff Mixter: +1 Harold Solbrig: +1 Tim Cole: +1 Ivan Herman: do we suspend all three PRs? Gregg Kellogg: not necessarily framing, but that needs its own discussion David Newbury: +1 Pierre-Antoine Champin: +1 Ivan Herman: +1 Gregg Kellogg: if there is no actual language in the API doc, the tests that have been added there might belong in the syntax doc Resolution #2: limit profile parameter use to URI’s, but continue to pursue an alternative to the profile media type parameter for client-signaled frames and contexts URLs for servers to (potentially) use when providing responses Benjamin Young: let’s leave it to gkellogg as to how he’d like to manage the PRs, whether to add more commits or start afresh or whatever |
|
I don't understand the resolution. The profile parameter can ONLY use:
According to the IANA registration of |
Exactly. 😃 The original proposal was to use them as URL's--extracting them from the profile value and dereferencing them and using them when parsing the data document. It was determined that client's requesting processing constraint documents (i.e. frames, contexts) was a potentially viable feature request, but that the @gkellogg mentioned using There will certainly need to be discussion about the use of I'd propose we change the title of this issue (s/parameters/header) to match reality. That sound OK? |
|
I'm 👎 on the resolution as I understand it. It conflates descriptive metadata and protocol transactions unnecessarily by applying rules intended for responses to the functionality of requests. Each entry in the Accept header is treated individually. It is perfectly reasonable to ask for json-ld or rdf/xml (syntax change) ; for json-ld or plain text (semantics are lost) ; or for two different flavors of json-ld. Those different flavors could be different contexts, which already imply an ontology selection according to that context's definition. If the context ever changes, then the semantics have changed, which according to the 100% strict reading, would not be acceptable and thus we would already be in an error state without introducing anything further usage. I'm also 👎 on changing parameter to header. Link headers are not a functional way to request content negotiable resources, as they lack all of the prioritization of q values and can't be grouped together with the media type they apply to. For example, say I want schema.org as JSON-LD, or simple Dublin Core as XML, because those are the two representation formats I have implemented. If the profile is separated from the media type, it would not be possible to determine that schema goes with json-ld, and DC goes with XML. Ergo, the profile MUST be a parameter and not in a separate header. And thus we either accept that it can go in profile, or we introduce a new parameter that quacks an awful lot like profile. |
|
@azaroth42 this has moved beyond content negotiation--which is I think where things are getting tangled and why it probably needs it's own issue(s). First, using Accept: application/ld+json;profile="http://www.w3.org/ns/anno.jsonld"and Accept: application/ld+json, application/ld+json;profile="http://www.w3.org/ns/anno.jsonld"still request that the server send back something that matches one of those (possibly profiled) types. However, this feature request included dereferencing the requested profile URL and processing that to generate the content--which per a stack of RFCs--goes against the intended use of the Consequently, There was not (and is not to my knowledge) any intention to replace or remove the use of Any clearer? 😄 |
|
@BigBlueHat is correct, the After closer inspection of the RFCs it seemed that our use of the However, your point about not being able to use a specific context URL as part of content negotiation is well taken, although it's not clear how a server would distinguish between different context or frame URLs to make a decision, without first dereferencing them, or using an internal registry. I'm not sure how we could allow for content negotiation on a context URL given the RFCs available to us. |
I don't see that anywhere in this issue, nor the referenced original issue. Can you provide a link? |
Hrm. It may have first come up at our TPAC conversations minuted above: #8 (comment) Which is why this is marked as having security concerns. There's no security concern if these are just opaque identifiers. |
|
It came up as a concern, certainly. Don't do that then, if there is a concern :) Just use a whitelisted set of profiles. IOW, there is no issue here about URI vs URL that isn't already addressed by an expanded security considerations section. |
|
From RFC 6906 3.0
This implies that an opaque URL is not appropriate for use in the profile, as, unless specifically registered, it cannot be dereferenced to affect the representation of the resource; at least, that was my take away from our discussion. This implies that our resolution, which @BigBlueHat mentioned from TPAC violates the stated purpose of the profile parameter. |
|
The next sentence, however clarifies that:
So I continue to disagree that there's anything wrong with our usage. |
@azaroth42 from what I can tell from the rest of the surrounding examples (mostly about podcasts), the intent is that at most a "dereferencable URI" (aka a URL) value for The use case of applying a profile seems a bit different than asking if the server has a resource conforming to a profile--hence the processing instruction comments from earlier. |
|
As an example, if I send However, it would be my expectation that the response would 406 if the only thing on disc were JSON(-LD) which did not conform to that profile (since I only asked for the profiled variant). The distinction seems important from a server implementation perspective, and having a means to intentionally request or initiate such processing does seem useful...but seems best signaled by something less implicitly inert. |
|
This issue was discussed in a meeting.
View the transcriptHTTP parameters for specifying context or frame (round 2!)Benjamin Young: #8 syntax - #111 (open) framing - w3c/json-ld-framing#34 (merged) api - w3c/json-ld-api#56 (ready for review) Benjamin Young: the framing PR is already merged — mostly clerical, regarding IANA registration. Gregg Kellogg: the PR is about the possibility for a client to specify the contex with which it wishes the retrieved data to be processed. … Last week, we discussed this, and realized that dereferencing the value of the profile attribute is something that we don’t want to do. … I had to rewrite this part. This information is now to be conveyed with a Link header. … A consequence is that the context URL can no more be used for content negotiation. Rob Sanderson: my understanding about the move away from the profile parameter is about the dereferencability? Gregg Kellogg: not exactly. The RFC indicates that the URI used as profile parameter can be dereferencable, for documentation. … But the execution should not be determined by derefencing it. Rob Sanderson: Why not simply specify that, if a server receives a profile URI that it does not know, … it just answers with 406? … We should not specify how the server does the serialization. … That’s implementation detail, not specification. … Given that, I don’t think there is any expectation that the server should dereference; … but it may do that if it thinks that’s a good thing. Benjamin Young: the spec last week was explicitly specifying that the profile parameter should be dereferenced. … Going away from this lead us away from the profile parameter all together. … But I see where you are going. Gregg Kellogg: We should have both mechanisms: one to query a well known profile, … and one to provide a previously unknown context, to be processed by the server. Rob Sanderson: we can’t prevent the profile from the accept header; it is part of the media-type. … We should embrace what people are going to do anyway, … which is a profile parameter in the accept header. Benjamin Young: I’m concerned we are defining a server API for JSON-LD, which is not what we are meant to do. Rob Sanderson: +1 to bigbluehat to steer away from JSON-LD-REST Rob Sanderson: (not because it’s a bad thing, but because we’re not chartered to do that!) Benjamin Young: We should provide a list of profile URI that we recommend. … Whatever else people use as profile URIs is their business. … The Links header is more an API spec, which would be LDP2’s job, not ours… Gregg Kellogg: there was always a problem for the profile parameter for, e.g., compacted … there was no way to convey the context against which compaction was done [not sure I got it right] … This has been long discussed in the CG. … The Link header was a way to address that. Benjamin Young: Is this purely an IANA registration and suggesting good practices? … I fear that too much HTTP matter in the spec and/or the test, … will feel like a MUST or a SHOULD for any JSON-LD implementation. Gregg Kellogg: that’s a way to ensure that people using it know how to use it. … The bit about the Link header could go in a different (REST?) spec or a best practices doc. Rob Sanderson: +1 to best practices tests Gregg Kellogg: We should remove any normative discussion about the Link header, … and replace it with a more vague mention to “header fields, to be described elsewhere”. … And move the current test elsewhere, probably a new spec. Proposed resolution: remove normative discussion of use of Link header usage for response augmentation–move to best practice for now (Benjamin Young) Ivan Herman: +1 Rob Sanderson: +1 Gregg Kellogg: +1 Benjamin Young: +1 Simon Steyskal: +1 Pierre-Antoine Champin: +1 Resolution #3: remove normative discussion of use of Link header usage for response augmentation–move to best practice for now Gregg Kellogg: Do we want to create new spec for server behaviour, or a best practices document? … And then should we create a different repository? Ivan Herman: I prefer many small repos, but I know not everyone agrees. … Wherever the editors feel it more conformable. … Should we close the issue? Gregg Kellogg: I still have a few changes to make. … After a small period for people to concur, … I’ll merge the changes. Ivan Herman: and then close the issue at the same time. |
ghost
removed
the
|
I'm coming in very late on this, but wrt a |
When requesting JSON-LD from an HTTP endpoint, it would be useful to provide a reference to a context or frame which should be used by the server to put the results into the proper format.
Original issue is JSON-LD 1.1 Feature Request : define how to specify the json-ld profile in a request to a server and include framing as an option #491
The text was updated successfully, but these errors were encountered: