Http profile

[gkellogg]

This adds tests for server content-negotiation on the profile parameter for the application/ld+json media type in an HTTP Accept header.

Not clear on what API text is necessary beyond the basic description of the profile parameter and its values in the syntax IANA section. Could conceivably add an HTTP Response section, describing the requirements for an HTTP server performing content-negotiation and how to interpret the parameters and other URLs present, but not sure that it's necessary.

For w3c/json-ld-syntax#8.

[msporny]

Interesting - looks like this is a PR to test if an HTTP endpoint is doing the appropriate processing on a JSON-LD document? Lots of files involved, hard for me to parse through it. Is there a link to the area of a spec that talks about all this (and any way the PR could be simplified to only the files affected... unless there are 49 of them!?).

[gkellogg]

There are, indeed, several files added, as there are 18 tests, each with 2-3 files, plus the files that define the test vocabulary, originally at json-ld.org/test-suite/vocab. The new test files are for a new "http-manifest", that specifically tests content negotiation using media-type profiles; tests/README.md discusses how to do the testing for HttpTest entries. The profiles are in the IANA section in the syntax doc, with a companion w3c/json-ld-syntax#111 that updates that.

No changes, as of yet, to the API index document, pending people's thoughts on what needs to be specified in that document. It's notable that most things in the API deal with retrieving documents and then performing an API method on them, while this would be for returning documents, including possible transformation along the way. We could add an HTTP Content Negotiation section, which could detail how to deal with different combinations of the defined profiles along with separate URL for context or frame, but I'm not sure that requires us to specify this.

[gkellogg]

Another thought: if there is, in fact, no need to add anything to the API document, we might move the HTTP tests to the syntax repository. We already have tests split between API and Framing repos.

[iherman]

This issue was discussed in a meeting.

• RESOLVED: limit profile parameter use to URI’s, but continue to pursue an alternative to the profile media type parameter for client-signaled frames and contexts URLs for servers to (potentially) use when providing responses

View the transcript

4.1. HTTP parameters for specifying context or frame
Benjamin Young: w3c/json-ld-syntax#8
Benjamin Young: * syntax - w3c/json-ld-syntax#111
Benjamin Young: * framing - w3c/json-ld-framing#34
Benjamin Young: * api - #56 (work in progress)
Benjamin Young: gkellogg has the lion’s share of the work done on this
Gregg Kellogg: the first PR (syntax doc, #111) pretty much ready to go
… removed language about quotes for the value of profile params
… which is not true in general
… only if multiple URIs are used, and added some examples with different types of profile params
Ivan Herman: I had a comment on that
… content, not editorial
… a sentence says that JSON-LD processors may restrict URIs for context and frames
… which I read as saying, “If I’m a JSON-LD processor, I can refuse any context except this particular one”
… is that really what we mean
Gregg Kellogg: well, this could become an attack vector, so we allow servers to restrict the action
Gregg Kellogg: framing is potentially expensive computationally
… and people have demonstrated attacks via context
… which we think we’ve mostly fixed, but obviously we can’t rule it entirely out
… going back to the CG, there was this desire to restrict present there
Ivan Herman: i have two problems…
… this is clearly non-editorial
… have we discussed/voted on this restriction?
… shouldn’t be in an editorial PR.
… I don’t understand the problem this solves
Gregg Kellogg: this is part of the request header.
Ivan Herman: oh, the request header? (not the response)
Gregg Kellogg: this signals the server as to what the client wants
… the server must examine the profile param and if it isn’t acceptable, return a 406
… this new language just gives license to the server to do just that if the context is unacceptable
… you could also fall-back in various ways
Ivan Herman: Okay, I misunderstood what was done, so that’s okay
… but still, this is more than editorial
Gregg Kellogg: I don’t say that this is editorial
… I folded in the previous discussions on this
Ivan Herman: sorry for the interrupt, fine with this. I will put in a link to this discussion into the PR
… must be documented, now it is
… we just want to document that everyone is fine with this
Benjamin Young: I’ve got a nit
… about the quotes being optional save for multiple URIs
Gregg Kellogg: for spaces
… and I show examples of this
… we don’t mandate that single spaces must be used
… it’s just whitespace as defined by the rFC
Benjamin Young: okay, thanks
Gregg Kellogg: we have also the PR for the API doc
Gregg Kellogg: #56
Gregg Kellogg: there is w-i-p on the API at that PR
… framing was a more minor change
Gregg Kellogg: w3c/json-ld-framing#34
Gregg Kellogg: the controversial point bigbluehat raised is whether adding a framing URI changes the semantics enough to make it inappropriate
… as a profile param
… online discussion indicates not
… the change requires the profile param
… which changes current behavior for people requesting frames
… there might b more changes requireed for this for optinality
Gregg Kellogg: the idea was that a frame would be represented by a mimetype
… but no one implemented that
… and now we’ve moved to using a profile param for frame
… so the question is, must the profile param appear, or is it optional/advisory
… this param is for getting a doc that is a frame
… and would appear on the response as well
Benjamin Young: yes, we would want that on the response as well
Benjamin Young: https://tools.ietf.org/html/rfc6906
Gregg Kellogg: we need more discussion on that PR
Adam Soroka: w3c/json-ld-framing#34
Gregg Kellogg: the API PR (#56 is just tests for this new behavior
… I couldn’t see what language to add to the API, otherwise
… the API do is concerned with what happens once you’ve requested a doc and are processing the result
… we don’t need to specify how to use an Accept header
Benjamin Young: this would only be for requesting a single doc in various representation
… I’m glad that framed docs aren’t new mediatypes
… but it seems unlikely that one document would be all the things at once (frame, context, instance)
Gregg Kellogg: but you aren’t requesting a doc, you are requesting a resource.
… e.g. for SPARQL users
Gregg Kellogg: profile contains one or more of our URIs, as well as URLs to contexts, frames, etc.
David Newbury: a real-world example: we have resources we’d like to be able to provide either as schema.org or CIDOC-CRM
… same data, difference context/framing
Gregg Kellogg: only the non-registered URLS would be treated as dereferencable
… similar to when the mimetype is just application/json and you stick the contet in a header
Benjamin Young: this is where I get concerned about violating the RFC– the profile param shouldn’t be dereferenced
Gregg Kellogg: good point…
… my impression was that we could define this behavior, but we do need to review
Benjamin Young: in the Web Annotation we avoided triggering new representations via the profile param, only selecting from extant representations
Ivan Herman: https://tools.ietf.org/html/rfc6906
Benjamin Young: this is a supernice feature, but is this where it belongs?
Ivan Herman: what the RFC (https://tools.ietf.org/html/rfc6906#section-3) does say is:
“A profile MUST NOT change the semantics of the resource representation when processed without profile knowledge, so that clients both with and without knowledge of a profiled resource can safely use the same representation.”
Benjamin Young: There is also https://tools.ietf.org/html/rfc4288#section-4.3
Ivan Herman: so this may indeed be a problem
Gregg Kellogg: the use of a context does not change the semantics of a representation, just the syntax
Benjamin Young: it’s the functionality concern
… if these are URLs (not merely URIs) they will be looked up and used for computation
Ivan Herman: it is an identifier; to quote further from https://tools.ietf.org/html/rfc6906#section-3:
“Profiles are identified by URI. However, as is the case with, for example, XML namespace URIs, the URI in this case only serves as an identifier,”
Ivan Herman: it is at the beginning of that section
Gregg Kellogg: rfc4288#section-4.3 is a different case of profile: that’s the header vs the mimetype
Gregg Kellogg: e.g. dc:name vs schema:name
… totally different graphs
… but for framing, it’s the same graph
… and client can reframe or re-compact or whatever
… for bigbluehat’s case, where there’s a range of possible contexts…
… two ways to interpret that: first, I want the Schema context vs the DC context
… or, I want “name” vs “schema:name” where the terms used are different
Benjamin Young: in the case of conneg, you could list several contexts
… and the sever will compute on your request and select something
… that’s not a processing instruction
… that’s a great idea, but it should go in a different box
Pierre-Antoine Champin: not fully agree with gkellogg about changing strings in the JSON for different contexts
… in both cases, the semantics are the same
… the condition for selecting different vocabularies is that the semantics of the vocabularies is the same
… in both cases, the changes are at the syntax level, just at concrete or abstract syntax
Gregg Kellogg: if we look at the test suite, what might be considered?
… one, you can take the result, expand it, and compare
… or two, turn them into RDF and compare the graphs for isomorphism
… (or in framing, a strict subgraph relationship)
Ivan Herman: +1 to gregg
David Newbury: I’m assuming that I have all the data and am just selecting from it, not doing new processing
Ivan Herman: whither shall we wander?
… what do we do if profiles are not the correct choice?
… our own request header?
… is there an extant header we can adopt?
Benjamin Young: Profile seems like the wrong vehicle for this signaling, but we cannot discard the use case
Proposed resolution: limit profile parameter use to URI’s, but continue to pursue an alternative to the profile media type parameter for client-signaled frames and contexts URLs for servers to (potentially) use when providing responses (Benjamin Young)
Gregg Kellogg: https://www.w3.org/TR/json-ld11/#interpreting-json-as-json-ld
Gregg Kellogg: we might borrow from the way we handle application/json
… in a response header, we add a link header targeting the context
… we could use a Link header on the request
Ivan Herman: we would need a more detailed description
Benjamin Young: https://tools.ietf.org/html/rfc8288
Benjamin Young: you sure can use a Link header on the request
… but let’s get to a proposal
… we need to get URLs out of the profiles
… and there is clear desire in the group for this functionality
… link rel= might be a nice way to do that
Proposed resolution: limit profile parameter use to URI’s, but continue to pursue an alternative to the profile media type parameter for client-signaled frames and contexts URLs for servers to (potentially) use when providing responses (Benjamin Young)
Gregg Kellogg: +1
Benjamin Young: +1
Adam Soroka: +1
Jeff Mixter: +1
Harold Solbrig: +1
Tim Cole: +1
Ivan Herman: do we suspend all three PRs?
Gregg Kellogg: not necessarily framing, but that needs its own discussion
David Newbury: +1
Pierre-Antoine Champin: +1
Ivan Herman: +1
Gregg Kellogg: if there is no actual language in the API doc, the tests that have been added there might belong in the syntax doc
Resolution #2: limit profile parameter use to URI’s, but continue to pursue an alternative to the profile media type parameter for client-signaled frames and contexts URLs for servers to (potentially) use when providing responses
Benjamin Young: let’s leave it to gkellogg as to how he’d like to manage the PRs, whether to add more commits or start afresh or whatever

[BigBlueHat]

Love the use of the Link header here. Feels very Webby. 😁 Guessing this approach will need its own set of syntax, api, and framing issues?

[gkellogg]

I'm not sure we need new issues. I've updated this PR with changes to use the Link header (although, we've discussed moving it to syntax) and w3c/json-ld-syntax#56 based on our discussions.

Framing needs an update, and we should consider not requiring a frame using the Link header and allowing a server to apply a default frame, as we do a default context.

[azaroth42]

I don't know what resolved issue this is addressing?

[BigBlueHat]

I don't know what resolved issue this is addressing?

This is why I was suggesting new issues. 😃 The cart seems to be a bit ahead of the horse--at least if you've missed the phone calls. We should get issues written up that more clearly express what's being solved for and why.

Also...I realize these are possible (in parts) "issues" express across our 3 repos...but (re)collecting them would help us all recollect them. 😜

[BigBlueHat]

@azaroth42 w3c/json-ld-syntax#8 (which you seem to have found) is likely the closest single issue on this topic--and the reasoning is in the call logs. But maybe that wasn't clear at first? or we need to sum it up better?

[azaroth42]

👍 to creating new issues. If it's unclear to me, now, it will be unclear to everyone in a year's time as to what we did and why

[iherman]

This issue was discussed in a meeting.

• RESOLVED: remove normative discussion of use of Link header usage for response augmentation–move to best practice for now

View the transcript

HTTP parameters for specifying context or frame (round 2!)
Benjamin Young: w3c/json-ld-syntax#8
syntax - w3c/json-ld-syntax#111 (open)
framing - w3c/json-ld-framing#34 (merged)
api - #56 (ready for review)
Benjamin Young: the framing PR is already merged — mostly clerical, regarding IANA registration.
Gregg Kellogg: the PR is about the possibility for a client to specify the contex with which it wishes the retrieved data to be processed.
… Last week, we discussed this, and realized that dereferencing the value of the profile attribute is something that we don’t want to do.
… I had to rewrite this part. This information is now to be conveyed with a Link header.
… A consequence is that the context URL can no more be used for content negotiation.
Rob Sanderson: my understanding about the move away from the profile parameter is about the dereferencability?
Gregg Kellogg: not exactly. The RFC indicates that the URI used as profile parameter can be dereferencable, for documentation.
… But the execution should not be determined by derefencing it.
Rob Sanderson: Why not simply specify that, if a server receives a profile URI that it does not know,
… it just answers with 406?
… We should not specify how the server does the serialization.
… That’s implementation detail, not specification.
… Given that, I don’t think there is any expectation that the server should dereference;
… but it may do that if it thinks that’s a good thing.
Benjamin Young: the spec last week was explicitly specifying that the profile parameter should be dereferenced.
… Going away from this lead us away from the profile parameter all together.
… But I see where you are going.
Gregg Kellogg: We should have both mechanisms: one to query a well known profile,
… and one to provide a previously unknown context, to be processed by the server.
Rob Sanderson: we can’t prevent the profile from the accept header; it is part of the media-type.
… We should embrace what people are going to do anyway,
… which is a profile parameter in the accept header.
Benjamin Young: I’m concerned we are defining a server API for JSON-LD, which is not what we are meant to do.
Rob Sanderson: +1 to bigbluehat to steer away from JSON-LD-REST
Rob Sanderson: (not because it’s a bad thing, but because we’re not chartered to do that!)
Benjamin Young: We should provide a list of profile URI that we recommend.
… Whatever else people use as profile URIs is their business.
… The Links header is more an API spec, which would be LDP2’s job, not ours…
Gregg Kellogg: there was always a problem for the profile parameter for, e.g., compacted
… there was no way to convey the context against which compaction was done [not sure I got it right]
… This has been long discussed in the CG.
… The Link header was a way to address that.
Benjamin Young: Is this purely an IANA registration and suggesting good practices?
… I fear that too much HTTP matter in the spec and/or the test,
… will feel like a MUST or a SHOULD for any JSON-LD implementation.
Gregg Kellogg: that’s a way to ensure that people using it know how to use it.
… The bit about the Link header could go in a different (REST?) spec or a best practices doc.
Rob Sanderson: +1 to best practices tests
Gregg Kellogg: We should remove any normative discussion about the Link header,
… and replace it with a more vague mention to “header fields, to be described elsewhere”.
… And move the current test elsewhere, probably a new spec.
Proposed resolution: remove normative discussion of use of Link header usage for response augmentation–move to best practice for now (Benjamin Young)
Ivan Herman: +1
Rob Sanderson: +1
Gregg Kellogg: +1
Benjamin Young: +1
Simon Steyskal: +1
Pierre-Antoine Champin: +1
Resolution #3: remove normative discussion of use of Link header usage for response augmentation–move to best practice for now
Gregg Kellogg: Do we want to create new spec for server behaviour, or a best practices document?
… And then should we create a different repository?
Ivan Herman: I prefer many small repos, but I know not everyone agrees.
… Wherever the editors feel it more conformable.
… Should we close the issue?
Gregg Kellogg: I still have a few changes to make.
… After a small period for people to concur,
… I’ll merge the changes.
Ivan Herman: and then close the issue at the same time.

Discussed on telecon.

[gkellogg]

Per resolution, these tests will not be added, and the PR is closed without merging.