Resource Path Discussion

[cmheazel]

One of the most contentious issues we face in API development is consensus on standard resource paths. This issue serves as an OGC-wide resource for continuing those discussions.

[pebau]

Hm, now we get a very intricate meaning of path components, based on the keywords. In our discussion yesterday we addressed this and thought that just leaving out everything after {collectionId} could act as a catch-all. Actually, I find this more straightforward.
But if we see items/ as the catch-all, then we need features/ as well to allow retrieving only the features, just as we can do only images, only coverages, etc.

my 2 morning cents,
Peter

[hylkevds]

Leaving out everything after {collectionId} (i.e. /collections/{collectionId}) will give you the metadata of the collection, not the data in the collection. So there has to be something behind the {collectionId} to get any data.

[cmheazel]

@hylkevds correct. /collections/{collectionId} returns the collectionInfo resource (metadata) for that collection. So we do need to define what comes after /items. Whatever it is, it needs to return the number matched, number returned, and next elements per API-Features. Which suggests something like a Feature Collection.

[jerstlouis]

If the same collection is available as vectors, coverage, and a map though, how to know what to return / expect?

[ilkkarinne]

I think it's important to distinguish cases where the (vector) features, coverages, maps, data/map tiles etc. are seen (by the data provider) as primary resources to access from the cases where they emerge as different representations of an underlying, also exposed common resource. So in the latter case you may have

• /collections/{collectionId}/items providing the items in a default representation,
• /collections/{collectionId}/items/map to provide them on a map,
• /collections/{collectionId}/items/map/tiles to provide them as set of map tiles,
• /collections/{collectionId}/items/coverage to provide them combined into a single coverage,
• /collections/{collectionId}/items/coverage/tiles to provide them combined into a coverage available as (data) tiles.

[cportele]

Leaving out everything after /collections/{collectionId} will give you the metadata of the collection, not the data in the collection.

Correct. /collections/{collectionId} returns the collectionInfo resource (metadata) for that collection.

Maybe it is splitting hairs, but it is important from a conceptual view: /collections/{collectionId} is the collection, not a metadata resource about the collection. Accessing the URI returns a representation of the collection, not a representation of metadata about the collection. For feature collections it is not practical or common practice to return the features which is why the representation includes a link with relation type items to a sub-resource that provides access to the items in the feature collection (/collections/{collectionId}/items). I.e., accessing /collections/{collectionId} does give access to the data in the collection, at least for collections with item type 'feature'.

[jerstlouis]

@cportele at least from the JSON representation perspective, /collections/{collectionId} is very much meta data about the collection, or at the very least a description of it (including some metadata). This is what describes the basic things like temporal and geospatial extents, and so on. But for an HTML representation you might want to show the collection and browse it right away in a nice online viewer, and that's fine...
In fact I was suggesting that /collections and /collections/{collectionId} could be the OGC API - Catalog (Core), which could be extended with search & filtering, and a /metadata for more detailed 19115 metadata.

For feature collections it is not practical or common practice to return the features

It is very common to be able to download shapefiles from ArcMap online, or GML from a WFS GetFeatures -- those are feature collections?

[joanma747]

@cportele, I have read the OGC API Features again. http://docs.opengeospatial.org/is/17-069r3/17-069r3.html#_collection_ In the current version, that could or could not return the data. The response description is so ambiguous that now I realize a full feature collection may also be a legal response!!!

I wonder the implications of this in OWS Common collections but I anticipate it as a "not good".
The scenario provided by @ilkkarinne is better for splitting the OWS Common part even I do not agree on some of the paths. To me, a scenario where a "CollectionId" returns some "generalities" and and links to get the actual data representations is easier to handle. The CollectionId level need another "word" to decided if you want to extract items or consider it as a continuous (or discrete) coverage, so path should look more like:

/collections/{collectionId}/items provides access to data fragmented into items (if the collection can be fragmented into items)
/collections/{collectionId}/map provides access to maps (and not as items)
/collections/{collectionId}/map/{styles}/tiles provides access to map tiles,
/collections/{collectionId}/tiles provides access to the data fragmented in tiles
/collections/{collectionId}/coverage provides the data as a single coverage,
/collections/{collectionId}/coverage/tiles provides a coverage fragmented into tiles (nobody has specified this in OGC yet)

/collections/{collectionId} is data in an "abstract" way and it cannot be directly retrievable without defining providing an "access" model first. And access modes can be multiple and you do not need to know with one is the "backend" one (even if it could be good to know).

[dblodgett-usgs]

I like what @cportele points out but think it's important to be clear -- the URI identifies the collection. The document you receive when dereferencing that URI is some representation of the entire collection -- which may look like "metadata". The key is that the top-level collection representation has a seamless path (through link rel "items") to expand the rest of the collections object.

In the SELFIE, we've pushed this distinction between "meta" resources and "data" resources a bit (too?) far. We (@abhritchie really) are just realizing that we need to step back and think about the distinction more carefully / with more people and technologies at the table. The distinction is useful from a communications and use-case definition standpoint but pushing it down into the technical specification or implementation is problematic -- as is discussed above.

[chris-little]

I would like to propose a slightly different interpretation on the URIs. In the soon to be active Environmental Data Retrieval API SWG, we envisage that paths like:
/collections/{collectionId}/point provides access to data at a (4D) location,
/collections/{collectionId}/timeseries provides access to data at a (3D) location over a time,
/collections/{collectionId}/verticalprofile provides access to data above/below a (2D) location at a time,
/collections/{collectionId}/trajectory provides access to data along a trajectory,
/collections/{collectionId}/corridor provides access to data within a corridor around and along a trajectory,
/collections/{collectionId}/polygon provides access to data within the polygon.

These 'items' or 'features' do not explicitly exist, except as transient queries with relevant search terms or metadata to the right of the '?'

We do not think these cause conflicts with other interpretatioin of the URIs

[cportele]

@jerstlouis

at least from the JSON representation perspective, /collections/{collectionId} is very much meta data about the collection, or at the very least a description of it (including some metadata). This is what describes the basic things like temporal and geospatial extents, and so on.

The current content reflects the consensus about the standard resource representations, but that doesn't change the fact that the resource is the collection itself.

In fact I was suggesting that /collections and /collections/{collectionId} could be the OGC API - Catalog (Core)

This is not the agreed scope of OGC API - Catalogues. I am not saying that things cannot be changed, but that would seem to be a significant change of scope and would require that the SWG would have to be recharted. This is also not my understanding of what the Catalogues API is about.

It is very common to be able to download shapefiles from ArcMap online, or GML from a WFS GetFeatures -- those are feature collections?

ArcGIS has a feature layer resource that is simlar to the OGC API Features collection resource and also separate resources to fetch the features.

WFS does not have a collection (or feature type) resource at all, it is RPC-based, not resource oriented and different requests return various bits and pieces of a collection, but the relationship of those bits is only implicit.

@joanma747

I have read the OGC API Features again. http://docs.opengeospatial.org/is/17-069r3/17-069r3.html#_collection_ In the current version, that could or could not return the data. The response description is so ambiguous that now I realize a full feature collection may also be a legal response!!!

It is not ambiguous. The requirements are all specified so that the current core content can be extended with additional information. This is essential. In this case, the description is clear that there has to be a link to fetch the features in the collection. If some server would add all the features in the collection resource representation (or any other information), yes, the server could do it, but why one do it? Generic clients would ignore this information and it would just be unnecessary noise. Please do not try to see issues where in practice there is no issue.

[jeffharrison]

The key thing to remember is... keep it simple.

Maybe I've missed some aspect of the dialog, but Resource paths should be easy to understand and implement. Like this...

The server SHALL support the HTTP GET operation at the path /styles

Done

[jerstlouis]

@jeffharrison The challenge is to try to ensure that the different APIs tie in together nicely.
And the '/collections' path, and /collections/{collectionId}/, is key to this because this is where the different geospatial data layers reside, and allows you to list these, and presumably do something with them (retrieve them in various ways, use them as input into processes, to render maps, and so on).

I personally believe it would be useful to take a few steps back from OpenAPI and "Resource paths" and think about how the different OGC API modules are meant to tie in together, before getting back to see what kind of resource paths could accommodate this.

[jeffharrison]

OGC APIs are (or should be ;-) modular, open building blocks. Easy to implement in days, not weeks. They can be aggregated if needed, or simply function on individual resources.

For example, if a user wants to reuse a style, they fetch a stylesheet of a style from

http://example.org/styles-api/styles/{styleId}

By the way, I've briefed this approach and people immediately understand it.

So as I read the dialog on this board it seems we are talking about two use cases... simple Resource paths and aggregated Resource paths.

I see no reason why it's a choice between one or the other.

[jerstlouis]

@jeffharrison I think the discussion is not about whether they should be simple or aggregated, but about figuring out the logical, intuitive ways how these simple resource paths should aggregate.

As well as coordination of the resource paths namespace between the different modules so that they are complementary, and not conflicting with each other.

[jeffharrison]

Jerome,

So let's do both - Define simple Resource paths and define aggregated Resource paths.

It's already underway and I see no problem. It's not one or the other, there's a place for both.

[dblodgett-usgs]

Totally agree, @jeffharrison. The easy button is a great trope to try and live up to and is worth including in what we are doing IMHO. There will also always be a need to pay the generalization penalty ensuring these APIs enable functionality that is reflective of the full scope of complexity of the data they represent and the scope of use cases people have for them.

The trick is that, inevitably, the easy button approach must be opinionated to limit the options and integrations it offers. I think what you are arguing for is to split these APIs apart to a sufficient degree that the number of opinionated decisions implicit in them is reduced to a minimum?

---

e.g. I just implemented a R function plot_nhdplus() (see visual examples here) that can be called as simply as

plot_nhdplus("05428500")

There is a significant group of people who will be thrilled with that. However, many will want to do much more so we can also do the same thing, "plot_nhdplus" in a few other ways:

plot_nhdplus(bbox = bbox)

plot_nhdplus(list(13293970, 13293750), streamorder = 3, nhdplus_data = local_data_path)

plot_nhdplus(list(list("comid", "13293970"),
                  list("nwissite", "USGS-05428500"),
                  list("huc12pp", "070900020603"),
                  list("huc12pp", "070900020602")),
             streamorder = 2,
             nhdplus_data = sample_data,
             plot_config = list(basin = list(lwd = 2),
                                outlets = list(huc12pp = list(cex = 1.5),
                                               comid = list(col = "green"))))

The reason I include this example is that -- I would have failed as a developer if I didn't provide what a huge number of people want plot_nhdplus("05428500") but that is simply not enough to stop people from writing this off as overly simplistic and going off to implement their own custom plots.

[jeffharrison]

Yes, I think. What we've been sponsoring in recent OGC Testbeds and Pilots is identifying geospatial Resources such as...

/tiles
/styles
/routes

...and associated APIs and encodings.

Participants in OGC Testbeds and Pilots have implemented these over the last year and shown they work. They've also been extensively documented in OGC Engineering Reports and Draft Specification ERs. When possible we've distributed demo videos on YouTube as well.

There will also certainly be more complex use cases that may require aggregated Resources too.

[hylkevds]

Path fragments that point to resources (.../tiles, .../styles, .../routes) can appear wherever they make sense. If I need to list all styles on a server, the path would be just /styles. Styles relevant for a certain collection would be fetched at /collections/{collectionId}/styles, and styles relevant for a certain map in a collection might appear in /collections/{collectionId}/maps/{mapId}/styles.

The question now becomes, what happens when multiple SWGs need to expose styles of incompatible types. Lets say we have styles for maps and styles for documents.

• Does /styles list both map styles and document styles? This would require a way to filter.
• Do we use a /mapStyles and a /docStyles? That is the clearest, but I'm sure both SWGs would prefer they get to use /styles.
• Does the first SWG that gets their standard approved get to claim /styles?
• Do we ignore the problem and hope it goes away (or never really becomes a problem in the first place)?

[cmheazel]

@hylkevds A path identifies a representation of a resource. So map and document representations of the collection should be on different paths. Something like:

/styles = styles that apply to the whole API
/collections/{collectionId}/styles = styles which apply to any representation of this collection
/collections/{collectionId}/maps/styles = styles which apply to the map representation of this collection
/collections/{collectionId}/maps/{mapId}/styles = styles which apply to this particular map representation.
/collections/{collectionId/doc/styles = styles which apply to the document representation of this collection.

We just have to agree on the datasets, representations, and encodings.

[cmheazel]

@hylkevds And since a style is a resource, we should be able to use the proposed CQL query language to filter styles.

[pebau]

Chuck, can you share more about the proposed CQL? thanks, Peter

On 27.11.19 13:41, Chuck Heazel wrote: @hylkevds <https://github.com/hylkevds> And since a style is a resource, we should be able to use the proposed CQL query language to filter styles. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#74>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABOR5DTXUF24TNNTYT5HXEDQVZTJLANCNFSM4JP5M4FA>.

-- Dr. Peter Baumann - Executive Director, rasdaman GmbH Bremen (HRB 26793) www.rasdaman.com, mail: baumann@rasdaman.com tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882 - Professor of Computer Science, Jacobs University Bremen www.faculty.jacobs-university.de/pbaumann mail: p.baumann@jacobs-university.de tel: +49-421-200-3178, fax: +49-421-200-493178 "A brilliant idea is a job halfdone."

[hylkevds]

I think users tend to expect contents of deeper paths to be a subset of shallower paths. So users expect .../styles to contain all data from both .../maps/styles and .../docs/styles. I'm not sure if that is what you mean or not, but it would lead to mixing different types of styles.

Filtering may work, but only if there is some clear property to filter on. That has to be taken into account when designing the data model, or we end up with very awkward queries. Two SWG that both design/expose a styles data model would have communicate to ensure such a filterable property exist in both their data models.

[cportele]

1. I am for an approach that resolves collisions when we identify them and that has a preference for simple, general terms in the API, where possible. We will not be able to define right away all kinds of resource types and their names that might be relevant for geospatial content in Web APIs in the future. With a mid-term horizon, the policy approved last week should give the mechanism to identify clashes that we think should be avoided before we enter the late stages of a draft standard and then this can be resolved.
   To use the "style" example, in the Styles API development we did not include a prefix like "map" because we were not aware of anyone using "style" for anything else in an OGC standard. It is SLD and not MSLD, for example. If there is a foreseeable clash with another resource type that would have the same name that will be identified and resolved in the process. If that results in "styles" being changed to "mapStyles" or "map-styles" that's ok.

2. That said, I do not think the names of path element names and collisions are the real issue. In my own Web API that also conforms to some OGC API standards I may still want to use /styles for some entirely different resource type that OGC will never write a standard about. This must be possible (we are not standardizing an OGC API, we are standardizing building blocks for reuse in Web APIs!) and clients have to be able to handle this. And they are. The fixed path patterns simplify using an API, if I know that the API supports certain OGC API conformance classes. The reverse, however, is not the case: the use of certain path element names in an API does not imply support for a certain OGC API conformance class. This is why we have the /conformance resource, for example. But more important from a general perspective is to play by the rules of the Web, for example, the definition and consistent (re-)use of link relation types and media types, proper API definitions, etc. The OGC API standards are not only for "us", the geo-community, they should make it easier to share or use geospatial content on the Web in general.
   Going back to the styles example, I think that in the Styles API we should use (and eventually try to register with IANA) a link relation type that is more specific. Currently we use rel=styles to link to /styles, but it probably should be something like rel=map-styles. I have not thought this to the end, but that is my current thinking.

[jerstlouis]

@cportele agreeing with everything you said there.

But I think it would be very worthwhile/useful to try to map the existing functionality of the core classic OGC services (specifically: WMS/WMTS, WFS, WCS, CSW and WPS) to new OGC API resource paths.

This would really provide a "big picture", and a great starting point for capabilities to aim for in a first version of the different standards, and make everyone confident that these key OGC API building blocks will all work great together, and at the same time clarify the transition from classic OGC services to the OGC API world.

I picture a nice diagram which could illustrate all this... A variant with the "classic services" mapping, and a lighter one without just showing how all of these building blocks functionality can be integrated in a single API.

[chris-little]

I agree with @cportele and to some extent @jerstlouis, but I would rather we started at the W3C HTTP(S) resources end. Then we might be able to identify those current/legacy client/server approaches that cannot or should not be promulgated into a resource orientated world.

[cmheazel]

@hylkevds Or you can look at the path depth as an indicator of the scope of the resource. A style defined at /collections/{collectionID} only applies to resources at or below this location on the path. OpenAPI uses this approach for Security Requirements and Parameters, so it should not come as a surprise to users.

The current version of API-Common distinguishes between spatial-temporal resources (which live in the collections) and information resources (which support the spatial-temporal resources). Spatial-temporal resources follow the pattern of sub-setting the deeper you go. Information resources follow the path defined scope pattern.

[cmheazel]

@pebau The CQL extension is here

[cmheazel]

There have been no comments on this issue since November 2019. The discussion has moved on to other issues and this one can be closed.

[ghobona]

I agree with #74 (comment). The discussion has moved on to other issues and this one can be closed.

[ghobona]

2020-08-24 SWG telecon resolution.

Relations should be there to be able to follow the links and that extensions may mandate the fixed path.

Action: @cmheazel To double check that this is already allowed or stated in the standard. If it is already stated, then @cmheazel can close this issue.

[ghobona]

2020-12-10 Cross SWG discussion

There was no further comment on this issue.

[jerstlouis]

@cmheazel This issue is now tagged as "close", but is still in the Backlog of the User Guide.
Has this been added to the User Guide that "Relations should be there to be able to follow the links and that extensions may mandate the fixed path."?
If the intent is simply not to block the Core release, maybe the Core label should just be taken out but leaving the issue open so it can be taken care of with the User Guide backlog?

Also, looking at the current content of the guide, I am wondering if some essential rules like this, or some of what is there right now, really doesn't belong in informative section of Part 1: Core, e.g. 6.3. Link Relations (which really is just Links in general) as also mentioned in #186 (comment), and 6.4 Relation Types.

[joanma747]

Decision to close it in 2021-02-22 telco