diff --git a/.github/workflows/Build-ShapeTopologies-spec.yml b/.github/workflows/Build-ShapeTopologies-spec.yml index d78d0b7..00b49b5 100644 --- a/.github/workflows/Build-ShapeTopologies-spec.yml +++ b/.github/workflows/Build-ShapeTopologies-spec.yml @@ -21,7 +21,7 @@ jobs: # if your doc isn’t in the root folder, # or Bikeshed otherwise can’t find it: - SOURCE: shape-topologies.bs + SOURCE: 02-shape-topologies.bs # output filename defaults to your input # with .html extension instead, diff --git a/.github/workflows/Build-TREE-spec.yml b/.github/workflows/Build-TREE-spec.yml index dd56a1b..350209f 100644 --- a/.github/workflows/Build-TREE-spec.yml +++ b/.github/workflows/Build-TREE-spec.yml @@ -21,7 +21,7 @@ jobs: # if your doc isn’t in the root folder, # or Bikeshed otherwise can’t find it: - SOURCE: spec.bs + SOURCE: 01-tree-specification.bs # output filename defaults to your input # with .html extension instead, diff --git a/.github/workflows/Build-discovery-spec.yml b/.github/workflows/Build-discovery-spec.yml new file mode 100644 index 0000000..9492308 --- /dev/null +++ b/.github/workflows/Build-discovery-spec.yml @@ -0,0 +1,29 @@ +name: Build TREE discovery spec +on: + workflow_dispatch: {} + pull_request: {} + push: + branches: [master] +jobs: + main: + name: Build, Validate and Deploy + runs-on: ubuntu-20.04 + permissions: + contents: write + steps: + - uses: actions/checkout@v3 + - uses: w3c/spec-prod@v2 + with: + TOOLCHAIN: bikeshed + + # Modify as appropriate + GH_PAGES_BRANCH: gh-pages + + # if your doc isn’t in the root folder, + # or Bikeshed otherwise can’t find it: + SOURCE: 03-discovery-specification.bs + + # output filename defaults to your input + # with .html extension instead, + # but if you want to customize it: + DESTINATION: discovery.html \ No newline at end of file diff --git a/01-tree-specification.bs b/01-tree-specification.bs new file mode 100644 index 0000000..63f071e --- /dev/null +++ b/01-tree-specification.bs @@ -0,0 +1,334 @@ +
+Title: The TREE hypermedia specification
+Shortname: TREE
+Level: 1
+Status: w3c/CG-DRAFT
+Markup Shorthands: markdown yes
+Group: TREE hypermedia community group
+URL: https://w3id.org/tree/specification
+Repository: https://github.com/treecg/specification
+Mailing List: public-treecg@w3.org
+Mailing List Archives: https://lists.w3.org/Archives/Public/public-treecg/
+Editor: Pieter Colpaert, https://pietercolpaert.be
+Abstract:
+    The TREE specification provides instructions for clients to interpret and navigate Web APIs structured as search trees.
+	It defines how members (sets of quads) in a dataset can be distributed across multiple pages interlinked through relationships.
+	The specification introduces key concepts such as `tree:Collection` (a set of members), `tree:Node` (the pages in the search tree), and `tree:Relation` (links between nodes).
+	By interpreting such qualified relations and search forms, TREE enables clients to efficiently retrieve their members of interest.
+
+ +# Overview # {#overview} + +An overview of the TREE specification with the TREE collection, a reference to the first focus node of its members, and the relations to other nodes from the current node. +
+The TREE specification introduces these core concepts: + * `tree:Collection` is a set of members. It typically has these properties when described in a node: + - `tree:member` points at the first focus node from which to retrieve and extract all quads of a member + - `tree:view` points to the `tree:Node` you are currently visiting + - `tree:shape` indicates the [[!SHACL]] shape (exactly one) to which each member in the collection adheres + * `tree:Node` is a page in the search tree + - `tree:relation` points at relations to other nodes + - `tree:search` describes a search form that allows an agent to jump from this node to a specific `tree:Node` in the (sub)tree + - `tree:viewDescription` points to an entity with a reusable piece of information relevant to the full search tree. Multiple descriptions MUST be combined. + * `tree:Relation` is a relation from one node to another. An extension of this class indicates a specific type of relation (e.g., a `tree:GreaterThanRelation`). A relation typically has these properties: + - `tree:node` is the URL of the other node + - `tree:path` indicates to which of the members' properties this relation applies + - `tree:value` indicates a value constraint on the members' values + - `tree:remainingItems` defines how many members can be reached when following this relation + +A simple collection can be created as illustrated in the following example: +
+ +
+ ```turtle + ex:Collection1 a tree:Collection; + rdfs:label "A Collection of subjects"@en; + tree:member ex:Subject1, ex:Subject2 . + + ex:Subject1 a ex:Subject ; + rdfs:label "Subject 1" ; + ex:value 1 . + + ex:Subject2 a ex:Subject ; + rdfs:label "Subject 2" ; + ex:value 2 . + ``` +
+ +From the moment this collection of members grows too large for one page, +a pagination needs to be created in which an initial set of members can be found through the first `tree:Node`, +and more members can be found by interpreting the TREE hypermedia controls. +This is illustrated in the next example: + +
+ ```turtle + > HTTP GET https://example.org/Node1 + + ex:Collection1 a tree:Collection; + tree:view ex:Node1 ; + tree:member ex:Subject1, ex:Subject2 . + + ex:Node1 a tree:Node ; + tree:relation ex:R1,ex:R2 . + + ex:R1 a tree:GreaterThanOrEqualToRelation ; + tree:node ex:Node3 ; # This is the URL of another page + tree:value 3; + tree:path ex:value . + + ex:R1 a tree:LessThanRelation ; # This is useful for a client that is looking for a value 10 or greater + tree:node ex:Node3 ; + tree:value 10; + tree:remainingItems 7 ; + tree:path ex:value . + + ex:R2 a tree:GreaterThanOrEqualToRelation ; + tree:node ex:Node4 ; + tree:value 10; + tree:remainingItems 10 ; + tree:path ex:value . + + ex:Subject1 a ex:Subject ; + rdfs:label "Subject 1" ; + ex:value 1 . + + ex:Subject2 a ex:Subject ; + rdfs:label "Subject 2" ; + ex:value 2 . + ``` +
+ +# Definitions # {#formalizations} + +A `tree:Collection` is a set of `tree:Member`s. +The set of members MAY be empty. + +A `tree:Member` is a set of (at least one) quad(s) defined by the member extraction algorithm (see further). + +A `tree:Node` is a dereferenceable resource containing `tree:Relation`s and a subset of (`⊆`) members of the collection. In a `tree:Node`, both the set of `tree:Relation`s as the subset of members MAY be empty. The same member MAY be contained in multiple nodes. + +A `tree:Relation` is a function denoting a conditional link to another `tree:Node`. + +A `tree:Node` is part of a search tree, and apart from the root node, it has exactly one other `tree:Node` of the search tree linking into it through one or more relations. + +A `tree:search` form is an IRI template, that when filled out with the right parameters becomes a `tree:Node` IRI, or when dereferenced will redirect to a `tree:Node` from which all members in the collection that adhere to the described comparator can be found. + +A search tree is the -- in this document -- implicit concept of a set of interlinked `tree:Node`s publishing a `tree:Collection`. +It will adhere to a certain growth or tree balancing strategy. +In one tree, completeness MUST be guaranteed, unless indicated otherwise (as is possible in LDES using a retention policy). + +# Initialization # {#init} + +A client SHOULD be initiated using a URL. +The client MUST dereference the URL, which will result in a set of [[!rdf-concepts]] triples or quads. +When the URL after all redirects, is used in a triple `?c tree:view <> .`, a client MUST assume the URL after redirects is an identifier of the intended root node of the collection in `?c`. + +Note: Dereferencing in this specification also means parsing the RDF triples or quads from the HTTP response. TREE does not limit the content-types that can be used to represent RDF triples. Client developers should do a best-effort for their community. + +If there is no such triple, then the client MUST check whether the URL before redirects (`E`) has been used in a pattern ` tree:view ?N.` where there’s exactly one `?N`, then the algorithm MUST return `?N` as the rootnode and `E` as the collection. + +The client then MUST dereference the identified rootnode (if it did not do that already) and merge those quads with the already found quads. +It now MUST look for a potential search forms that MAY be linked, either i) on top of the rootnode, or ii) on top of the entity linked through `tree:viewDescription`, using `tree:search`. + +In case it is not done using an unambiguous URL, clients MAY implement the report on [Discovery and Context Information (work in progress)](https://w3id.org/tree/specification/discovery). +This report also explains how clients MAY implement support for extracting context information such as provenance, contact points, etc. + +Note: Having an identifier for the collection has become mandatory: without it you can otherwise not define completeness. + +# The member extraction algorithm # {#member-extraction-algorithm} + +The member extraction algorithm allows a data publisher to define their members in different ways: + 1. As in the examples above: all quads with the object of the `tree:member` quads as a subject (and recursively the quads of their blank nodes) are by default included (see also [[!CBD]]), except when they would explicitly not be included in case 3, when the shape would be closed. + 2. Out of band / in band: + - when no quads of a member have been found, the member will be dereferenced. This allows to publish the member on a separate page. + - part of the member can be maintained elsewhere when a shape is defined (see 3) + 3. By defining a more complex shape with `tree:shape`, also nested entities can be included in the member + 4. By putting the triples in a named graph of the object of `tree:member`, all these triples will be matched. + +Depending on the goals of the client, it MAY implement the member extraction algorithm to fetch all triples about the entity as intended by the server. +The method used within TREE is combination of Concise Bounded Descriptions [[!CBD]], named graphs and the topology of a shape (deducted from the `tree:shape`). +The full algorithm is specified in the [shape topologies](https://w3id.org/tree/specification/shape-topologies) report. + +# Traversing the search tree # {#traversing} + +After dereferencing a `tree:Node`, a client MUST extract all (zero or more) `tree:Relation` descriptions from the page. +This can be done by searching for `<> tree:relation ?R` triples. + +A client MUST follow the object of the relation’s `?R tree:node ?object` triple, unless the client is able to prune the branch reachable from that node (see further). + +A client MAY also extract the `tree:Relation`’s `tree:remainingItems` if it exists. +If it does, it will be an integer indicating the remaining items to be found after dereferencing the node. + +When traversing, a client SHOULD detect faulty search trees by keeping a list of already visited pages. + +When dereferencing the object of a `tree:node` triple, the client MUST follow redirects. + +Note: Allowing redirects allows servers to rebalance their search trees over time. + +A client can assume completeness of members intended by the search tree when it derefenced all node links. + +# Pruning branches # {#relationsubclasses} + +In search trees, a `tree:Relation` will likely be typed using one of its subclasses: + * For partial string matching, `tree:PrefixRelation`, `tree:SubstringRelation`, and `tree:SuffixRelation` exist. + * For comparing various datatypes, `tree:GreaterThanRelation`, `tree:GreaterThanOrEqualToRelation`, `tree:LessThanRelation`, `tree:LessThanOrEqualToRelation`, `tree:EqualToRelation`, and `tree:NotEqualToRelation` exist. + * Finally, for geospatial trees, `tree:GeospatiallyContainsRelation` exists. + +A client decides, based on their own tasks, what type of relations are important to implement. +Each relation is a comparator function that helps deciding whether or not the subtree reachable from the `tree:node` link can be pruned. +A relation can be interpreted as a comparator as follows: + 1. The left-hand: what the members in the subtree reachable from the linked node will contain w.r.t. the objects reachable from the `tree:path`. + 2. The operator: decided by the type of the relation and the datatype or node type of the `tree:value` triple’s object. + 3. The right-hand: the `tree:value` triple’s object. + +The client MUST combine all relations to the same `tree:node` using a logical AND. + +The members that the client is able to find in a subtree will be complete relative to the position in the search tree. + +
+```turtle +<> tree:relation [ + a tree:GreaterThanRelation ; # the type of the relation deciding the operator + tree:node ex:Node2 ; # for the left-hand: all members from here + tree:path dct:created ; # for the left-hand: the path pointing at the term(s) in the member + tree:value "2024-12-16T12:00:00Z"^^xsd:dateTime # the right-hand + ],[ + a tree:SubstringRelation ; + tree:node ex:Node2 ; + tree:path dct:title ; + tree:value "osa" + ] . +``` +
+ +
+In the example above the subtree reachable from `ex:Node2` will contain all remaining members that are both created later in time than the given timestamp *and* will have the provided substring in the title. +The client can choose to prune all links to other nodes if this is the only thing it is interested in. +Alternatively, the client can choose prune the subtree reachable from `ex:Node2` if it is specifically not looking for members with the given substring, *or* when it is not interested in members created later in time than the given timestamp. +Alternatively, it can also score the relation based on the likelihood of returning useful results and created a priority queue that is processed until a top K of results have been found. +Mind that when the client is specifically not interested in members created later than the given creation time, but does not understand the SubstringRelation, the client can still prune the relation. +
+ +While each type of relation can decide on their own properties, +relations will often use the `tree:path` to indicate the path from the member to the object on which the `tree:Relation` applies. +For the different ways to express or handle a `tree:path`, we refer to [2.3.1 in the shacl specification](https://www.w3.org/TR/shacl/#x2.3.1-shacl-property-paths). +All possible combinations of e.g., `sh:alternativePath` or `sh:inversePath` in the SHACL spec can be used. +The resulting values of the evaluation of the `tree:path`, are the values that must be compared to the `tree:value` object. +When multiple results from the path are found, they need to be interpreted as a logical OR: at least one of these values will fulfill the comparator. + +A client, in case it wants to process relations that use the `tree:path` property, MUST implement a matching algorithm to check whether the relation is relevant. +I.e., a `tree:path` on `(rdfs:label [sh:alternativePath rdfs:comment ] )` will be useful when the client is tasked to filter on `rdfs:comment`. + +Note: A server is allowed to refer the `tree:path` to a property that is not materialized in the current response. For the client, if it also needs those triples, we assume in this spec that the client has another way of retrieving those, or already retrieved them from another source. + +## Comparing strings ## {#strings} + +String values have three specific type of relations: the `tree:PrefixRelation`, the `tree:SubstringRelation` and the `tree:SuffixRelation`. +The string comparison happens using the unicode canonical equivalence. + +Issue: We experimented with server-chosen locales such that `ça suffit` can also be found when following a `tree:PrefixRelation` with a `tree:value "c"` (which at this moment is not supported). That would require an understanding of locales, and [browser/JavaScript support for locales is too low to be useful at this point](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl#Locale_identification_and_negotiation). + +Also the comparator relations such as `tree:GreaterThanRelation` can be used. +The strings MUST then be compared using case sensitive unicode ordering. + +When a language is set on the `tree:value`, the relation only refers to these language strings. If no language is indicated, it refers to all (including those without). + +Particular to the `tree:SubstringRelation`, is that multiple `tree:value` properties can be set. This means the members properties will contain all of the given substrings. + +Issue: We need a flag for setting case insensitiveness: what will we use? In previous implementations `sh:flags "i"` was used. + +## Comparing named nodes ## {#named-nodes} + +When using comparator relations such as `tree:GreaterThanRelation`, named nodes MUST be compared as defined in the [ORDER BY section of the SPARQL specification](https://www.w3.org/TR/sparql11-query/#modOrderBy). + +## Comparing geospatial features ## {#geospatial} + +The `tree:GeospatiallyContainsRelation` is the relation that can be used to express all further members will be contained within a geospatial region defined by the WKT String in the `tree:value`. +The client MUST consider the relation when it overlaps with the region of interest. + +When using `tree:GeospatiallyContainsRelation`, the `tree:path` MUST refer to a literal containing a WKT string, such as `geosparql:asWKT`. + +## Comparing time literals ## {#time} + +When using relations such as `tree:LessThanRelation` or `tree:GreaterThanRelation`, the time literals MUST to be compared according to these 3 possible data types: `xsd:date`, `xsd:dateTime` or `xsd:dateTimeStamp`. +It is highly recommended to server developers to provide a timezone in the `tree:value`, which can be done in these datatypes themself. + +When no timezone is specified, the comparison needs to take place on the worst-case bound. For example a date `2022-01-01` without timezone thus represents a period of time of 48 hours from (`[`) `2021-12-31T12:00:00Z` until `2022-01-02T12:00:00Z` (`[`). + +# Search forms # {#searching} + +Searching through a TREE will allow you to immediately jump to the right `tree:Node` in a subtree. +TREE relies on the [Hydra search specification](http://www.hydra-cg.com/spec/latest/core/#hydra:search) for its search forms. +It does however extend Hydra with specific search properties (`hydra:IriTemplate`) for different types of search forms, and searches starting from a specific `tree:Node`, to which the search form is linked with `tree:search`. +The behaviour of the search form fully depends on the specific property, for which TREE introduces a couple of specific properties: + +## Geospatial XYZ tiles search form ## {#xyztiles} + +Three properties allow to specify a geospatial XYZ tiles template (also known as slippy maps). + 1. `tree:longitudeTile` describes the X value + 2. `tree:latitudeTile` describes the Y value + 3. `tree:zoom` describes the zoom level + +All properties expect positive integers. + +
+ ```turtle + a tree:Collection ; + dcterms:title "A prototype tree:Collection for Linked OpenStreetMap’s roads"@en . + a tree:Node ; + + tree:search [ + a hydra:IriTemplate ; + hydra:template "https://tiles.openplanner.team/planet/20201103-095900/{z}/{x}/{y}" ; + hydra:variableRepresentation hydra:BasicRepresentation ; + hydra:mapping [ + a hydra:IriTemplateMapping ; + hydra:variable "x"; + hydra:property tree:longitudeTile; + hydra:required true + ],[ + a hydra:IriTemplateMapping ; + hydra:variable "y"; + hydra:property tree:latitudeTile; + hydra:required true + ],[ + a hydra:IriTemplateMapping ; + hydra:variable "z"; + hydra:property tree:zoom; + hydra:required true + ] + ] . + ``` +
+ +This search form describes a specific search form that uses a quad tree. The zoom level describes the depth, the longitudeTile and latitudeTile describe the x and y index of the pagination. (e.g., on zoom level 0, there’s 1 tile, on zoom level 1, there are 4 tiles, etc.). + +## Searching through a list of objects ordered by time ## {#timesearch} + +Same as the previous example but with the predicate `tree:timeQuery` expecting an `xsd:dateTime`. +This time however, when the page itself does not exist, a redirect is doing to happen to the page containing the timestamp. +A `tree:path` can indicate the time predicate which is intended. + +
+ ```turtle + a tree:Collection ; + dcterms:title "An example collection with a time search view"@en ; + tree:view . + + a tree:Node ; + tree:search [ + a hydra:IriTemplate ; + hydra:template "https://example.org/{generatedAt}" ; + hydra:variableRepresentation hydra:BasicRepresentation ; + hydra:mapping [ + a hydra:IriTemplateMapping ; + hydra:variable "generatedAt"; + tree:path prov:generatedAtTime; + hydra:property tree:timeQuery; + hydra:required true + ] + ] . + ``` +
+ +
path: vocabulary.md
diff --git a/shape-topologies.bs b/02-shape-topologies.bs similarity index 100% rename from shape-topologies.bs rename to 02-shape-topologies.bs diff --git a/03-discovery-specification.bs b/03-discovery-specification.bs new file mode 100644 index 0000000..751eddb --- /dev/null +++ b/03-discovery-specification.bs @@ -0,0 +1,124 @@ +
+Title: TREE Discovery and Context Information
+Shortname: TREEDiscovery
+Level: 1
+Status: w3c/CG-DRAFT
+Markup Shorthands: markdown yes
+Group: TREE hypermedia community group
+URL: https://w3id.org/tree/specification/discovery
+Repository: https://github.com/treecg/specification
+Mailing List: public-treecg@w3.org
+Mailing List Archives: https://lists.w3.org/Archives/Public/public-treecg/
+Editor: Pieter Colpaert, https://pietercolpaert.be
+Abstract:
+    This specification defines how a client selects a specific dataset and search tree, as well as extracts relevant context information.
+
+ +# Definitions # {#overview} + +A `tree:Collection` is a subclass of `dcat:Dataset` ([[!vocab-dcat-3]]). +The specialization being that this particular dataset is a collection of _members_. + +A `tree:SearchTree` is a subClassOf `dcat:Distribution`. +The specialization being that it uses the main TREE specification to publish a search tree. + +A node from which all other nodes can be found is a `tree:RootNode`. + +Note: The `tree:SearchTree` and the `tree:RootNode` MAY be identified by the same IRI when no disambiguation is needed. + +A TREE client MUST be provided with a URL to start from, which we call the _entrypoint_. + +# Initializing a client with a url # {#starting-from} + +The goal of the client is to understand what `tree:Collection` it is using, and to find a `tree:RootNode` to start the traversal phase from. +This discovery specification extends the initialization step in the TREE specification, for the cases in which multiple options are possible. + +The client MUST dereference the URL, which will result in a set of quads. The client now MUST first perform the init step from the main specification. +If that did not return any result, then the client MUST check whether the URL before redirects (`E`) has been used in one of the following discovery patterns described in the subsections: + 1. `E` is a `tree:Collection`: then the client needs to [select the right search tree](#tree-search-trees) + 2. `E` is a `dcat:Dataset`: then the client needs to [select the right distribution or dataservice from a catalog](#dcat-dataset) + 3. `E` is a `ldes:EventStream`: then the client MAY take into account [LDES specific properties](#ldes) + 4. `E` is a `dcat:Distribution`: then the client needs to [process it accordingly](#dcat-distribution) + 5. `E` is a `dcat:DataService`: then the client needs to [process it accordingly](#dcat-dataservice) + 6. `E` is a catalog or is not explicitly mentioned: then it needs to select a dataset based on [shape information](#tree-collection-shapes) and [DCAT Catalog information](#dcat-catalog) + +## Selecting a collection via shapes ## {#tree-collection-shapes} + +When multiple collections are found by a client, it can choose to prune the collections based on the `tree:shape` property. +The `tree:shape` property will refer to a first `sh:NodeShape`. +The collection MAY be pruned in case there is no overlap with the properties the client needs. + +Issue: Will we document the precise algorithm to use? Should we extend shapes with cardinality approximations as well? + +## Selecting a collection via a catalog ## {#dcat-catalog} + +A DCAT Catalog is an overview of datasets, data services and distributions. +As TREE clients first need to select a dataset, and then a search tree to use, it aligns with how DCAT-AP works. +DCAT discovery extends upon the previous section in which a collection or dataset can be selected based on the `tree:shape` property. + +For now, we will assume the DCAT information is available in subject pages. + +Issue: Do we need more text on how to handle different types of DCAT interfaces? + +The dataset descriptions can be used for filtering the datasets available in a catalog to a list of datasets that can be useful for the client. +Such properties may include the spatial extent, the time extent, or how it is possibly a part of another `dcat:Dataset`. + +Issue: How precise do we need to be in this specification? + +When the `dcat:Dataset` is a `tree:Collection`, the DCAT catalog is going to contain a `dct:type` property with `https://w3id.org/tree#Collection` or `https://w3id.org/ldes#EventStream` as the object. + +## Choosing from multiple SearchTrees with TREE ## {#tree-search-trees} + +Issue: This is yet to be done + +## Selecting a search tree via a DCAT dataset ## {#dcat-dataset} + +The are two ways in which you can find a search tree from a dataset: via the distributions and via the data services. Both need to be tested. +Selecting a distribution or data service when multiple are available needs to be done based on [the search tree description](tree-search-trees). +If nothing is available, all need to be tested by processing them as exemplifie din the next subsections. + +### Selecting a search tree via DCAT Distribution ### {#dcat-distribution} + +`E dcat:distribution ?D . ?D dcat:downloadURL ?N .` then ?N is a rootnode of E. + +Issue: This is yet to be done + +### Selecting a search tree from a DCAT data service ### {#dcat-dataservice} + + * `?DS dcat:servesDataset E ; dcat:endpointURL ?U` or `E dcat:endpointURL ?U`, then the algorithm MUST repeat the algorithm with `?U` as the entrypoint. + +Issue: This is yet to be done + +## Linked Data Event Streams ## {#ldes} + +In case the client is not made for query answering, but only for setting up a replication and synchronization system, then there is a special type that can be used to indicate the search tree is made for this purpose: the `ldes:EventSource`. +Clients that want to prioritize taking a _full_ copy MAY give full priority to this server hint. + +
+```turtle +E a ldes:EventSource ; + tree:rootNode|dcat:downloadURL . +``` +
+ +# Extracting content information # {#context} + +Issue: This is yet to be done + +Context information enables a client to understand who the creator of a certain dataset is, when it was last changed, what other datasets it was derived from, etc. + +## DCAT and dcterms ## {#context-dcat} + +Issue: This is yet to be done + +## Provenance ## {#context-prov} + +Issue: This is yet to be done + +## Linked Data Event Streams ## {#context-ldes} + +Issue: This is yet to be done + +LDES (https://w3id.org/ldes/specification) is a way to evolve search trees in a consistent way. It defines every member as immutable, and a collection as append-only. +Therefore, one can make sure to only process each member once. +Extra terms are added, such as the concept of an EventStream, retention policies and a timestampPath. diff --git a/TREE-overview.svg b/TREE-overview.svg new file mode 100644 index 0000000..91d6adf --- /dev/null +++ b/TREE-overview.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/specs/4-provenance-and-summaries.md b/drafts/4-provenance-and-summaries.md similarity index 100% rename from specs/4-provenance-and-summaries.md rename to drafts/4-provenance-and-summaries.md diff --git a/specs/6-compatibility.md b/drafts/5-compatibilities.md similarity index 92% rename from specs/6-compatibility.md rename to drafts/5-compatibilities.md index 3904bff..658c43d 100644 --- a/specs/6-compatibility.md +++ b/drafts/5-compatibilities.md @@ -1,9 +1,5 @@ # Compatibility # {#compatibility} -## DCAT ## {#dcat} - -[[!VOCAB-DCAT-2]] is the standard for Open Data Portals by W3C. In order to find TREE compliant datasets in data portals, there SHOULD be a dcat:endpointDescription from the dcat:DataService to the entrypoint where the tree:Collections and the tree:ViewDescriptions are listed. Furthermore, there SHOULD be a dct:conformsTo this URI: https://w3id.org/tree/specification. - ## Hydra ## {#hydra} A tree:Collection is compatible with the [Hydra Collections specification](https://www.hydra-cg.com/spec/latest/core/#collections). However, instead of hydra:view, we use tree:view and do not link to a hydra:PartialCollectionView but to a tree:Node. diff --git a/specs/5-imports.md b/drafts/6-imports.md similarity index 100% rename from specs/5-imports.md rename to drafts/6-imports.md diff --git a/examples/eventstreams/README.md b/examples/eventstreams/README.md deleted file mode 100644 index 4caf2f5..0000000 --- a/examples/eventstreams/README.md +++ /dev/null @@ -1,82 +0,0 @@ -# Example: Event Streams - -Event streams in TREE are defined as a `tree:Collection` of members with event-like objects. Each of such member has a timestamp (preferably with range `xsd:dateTime`) such as a `prov:generatedAtTime` indicating the time at which the event was created. -On each collection, a `tree:shape` __should__ be set that links to a SHACL shape to which the events comply. - -Example: - -```turtle - a tree:Collection ; - tree:shape ; # this shacl shape for as long as this collection exists will need to be backwards compatible. - tree:member . - - a sosa:Observation ; - sosa:resultTime "2020..." ; - sosa:hasSimpleResult "1" . -``` - -When implementing an event stream on top of a data model that does not have the concept of things that live in time, you will have to extend that model. One can solve that by using the concept of versions (using the predicate `dcterms:isVersionOf` in dcterms as in the following example: - -```turtle - a tree:Collection ; - tree:shape ; - tree:member . - - prov:generatedAtTime "2021-01-01T00:00:00Z" ; - adms:versionNotes "First version of this address. It was added in 2021." ; - dcterms:isVersionOf ; - dcterms:title "Streetname X, ZIP Municipality, Country" . -``` - -A SPARQL query then has to query for version of this object as a way to give time-context to the query: - -```sparql -SELECT * WHERE { - ?address dcterms:isVersionOf ; - prov:generatedAtTime ?created ; - dcterms:title ?addressString . -} ORDER BY DESC ?created LIMIT 1 -``` - -A CONSTRUCT query can thus materialize the last version as follows: - -```sparql -CONSTRUCT { - dcterms:title ?addressString . -} WHERE { - ?address dcterms:isVersionOf ; - prov:generatedAtTime ?created ; - dcterms:title ?addressString . -} ORDER BY DESC ?created LIMIT 1 -``` - -## Fragmentation strategies - -The main interesting fragmentation strategy for an event stream is to split it in time. -This can be done with simple TREE relations such as: `tree:GreaterThanOrEqualToRelation` and `tree:LessThanOrEqualToRelation`. - -The first page of the event stream starts with the oldest events: - -```turtle - a tree:Collection ; - tree:shape ; # this shacl shape for as long as this collection exists will need to be backwards compatible. - tree:member , ... ; - tree:view . - - a tree:Node ; - tree:relation [ - a tree:GreaterThanOrEqualToRelation ; - tree:path sosa:resultTime ; - tree:node ; - tree:value "2020-12-24T12:00:00Z"^^xsd:dateTime - ] . -``` - -Also other links can be added to later pages. - -A `tree:importStream` can be used on the last page to import events through pubsub as they happen in time. - -## Hydra search forms - -Using a `hydra:search` form with `hydra:property` `tree:timeQuery` you may add a search form to directly address a certain page containing events in a time interval. - diff --git a/examples/geospatially-ordered-public-transport/first.ttl b/examples/geospatially-ordered-public-transport/first.ttl deleted file mode 100644 index f9f5374..0000000 --- a/examples/geospatially-ordered-public-transport/first.ttl +++ /dev/null @@ -1,58 +0,0 @@ -@prefix tree: . -@prefix hydra: . -@prefix dcterms: . -@prefix rdfs: . -@prefix schema: . -@prefix shacl: . -@prefix lc: . -@prefix gtfs: . -@prefix geosparql: . -@prefix xsd: . - - a hydra:Collection ; - rdfs:label "All arrivals and departures of Trains in Belgium"@en ; - tree:view . #From the first page, all elements can be found - - a tree:Node ; - tree:relation [ - a tree:GreaterThanOrEqualToRelation ; - tree:node ; - tree:path lc:departureTime ; - tree:value "2020-03-16T08:30:00.000Z"^^xsd:dateTime - ], - [ - a tree:GeospatiallyContainsRelation ; - tree:node ; - tree:path (lc:departureStop geosparql:asWKT) ; - tree:import ; - tree:value "POLYGON ((30 10, 40 40, 20 40, 10 20, 30 10))"^^geosparql:wktLiteral - ]. - - hydra:member , - . - -## The data itself - a lc:Connection; - lc:arrivalDelay "240"^^xsd:integer ; - lc:arrivalStop ; - lc:arrivalTime "2020-03-16T08:45:00.000Z"^^xsd:dateTime ; - lc:departureDelay "360"^^xsd:integer ; - lc:departureStop ; - lc:departureTime "2020-03-16T08:28:00.000Z"^^xsd:dateTime ; - gtfs:dropOffType gtfs:Regular; - gtfs:headsign "Ostende"; - gtfs:pickupType gtfs:Regular; - gtfs:route ; - gtfs:trip . - a lc:Connection; - lc:arrivalDelay "180"^^xsd:integer ; - lc:arrivalStop ; - lc:arrivalTime "2020-03-16T08:56:00.000Z"^^xsd:dateTime ; - lc:departureDelay "180"^^xsd:integer ; - lc:departureStop ; - lc:departureTime "2020-03-16T08:29:00.000Z"^^xsd:dateTime ; - gtfs:dropOffType gtfs:Regular ; - gtfs:headsign "Bruges" ; - gtfs:pickupType gtfs:Regular ; - gtfs:route ; - gtfs:trip . \ No newline at end of file diff --git a/examples/geospatially-ordered-public-transport/second.ttl b/examples/geospatially-ordered-public-transport/second.ttl deleted file mode 100644 index d16effa..0000000 --- a/examples/geospatially-ordered-public-transport/second.ttl +++ /dev/null @@ -1,42 +0,0 @@ -@prefix tree: . -@prefix hydra: . -@prefix dcterms: . -@prefix rdfs: . -@prefix schema: . -@prefix shacl: . -@prefix lc: . -@prefix gtfs: . -@prefix xsd: . - - a hydra:Collection ; - rdfs:label "All arrivals and departures of Trains in Belgium"@en ; - tree:view ; - hydra:member , - . - -## The data itself - - a lc:Connection; - lc:arrivalDelay "60"^^xsd:integer; - lc:arrivalStop ; - lc:arrivalTime "2020-03-16T09:12:00.000Z"^^xsd:dateTime; - lc:departureDelay "180"^^xsd:integer; - lc:departureStop ; - lc:departureTime "2020-03-16T08:32:00.000Z"^^xsd:dateTime; - gtfs:dropOffType gtfs:Regular; - gtfs:headsign "Bruges"; - gtfs:pickupType gtfs:Regular; - gtfs:route ; - gtfs:trip . - a lc:Connection; - lc:arrivalDelay "0"^^xsd:integer>; - lc:arrivalStop ; - lc:arrivalTime "2020-03-16T09:02:00.000Z"^^xsd:dateTime; - lc:departureDelay "0"^^xsd:integer; - lc:departureStop ; - lc:departureTime "2020-03-16T08:36:00.000Z"^^xsd:dateTime; - gtfs:dropOffType gtfs:Regular; - gtfs:headsign "Zottegem"; - gtfs:pickupType gtfs:Regular; - gtfs:route ; - gtfs:trip . \ No newline at end of file diff --git a/examples/geospatially-ordered-public-transport/stops.ttl b/examples/geospatially-ordered-public-transport/stops.ttl deleted file mode 100644 index ea87971..0000000 --- a/examples/geospatially-ordered-public-transport/stops.ttl +++ /dev/null @@ -1,12 +0,0 @@ -@prefix rdfs: . -@prefix gtfs: . -@prefix geo: . -@prefix geosparql: . -@prefix xsd: . - - - a gtfs:Stop ; - rdfs:label "Izegem"@nl ; - geo:latitude 50.9211500 ; - geo:longitude 3.21208900 ; - geosparql:asWKT "POINT (3.21208900 50.9211500)"^^geosparql:wktLiteral . diff --git a/examples/paged-collection-with-order/first.ttl b/examples/paged-collection-with-order/first.ttl deleted file mode 100644 index 68e9589..0000000 --- a/examples/paged-collection-with-order/first.ttl +++ /dev/null @@ -1,48 +0,0 @@ -@prefix tree: . -@prefix hydra: . -@prefix dcterms: . -@prefix rdfs: . -@prefix schema: . -@prefix shacl: . -@prefix lc: . -@prefix xsd: . - - a hydra:Collection ; - rdfs:label "All arrivals and departures of Trains in Belgium"@en ; - tree:view . #From the first page, all elements can be found - - a tree:Node ; - tree:relation [ - a tree:GreaterThanRelation ; - tree:node ; - tree:path lc:departureTime ; - tree:value "2019-07-23T06:30:00.000Z"^^xsd:dateTime - ]. - - hydra:member , . - -## The data itself - a lc:Connection; - lc:arrivalDelay "240"^^; - lc:arrivalStop ; - lc:arrivalTime "2019-07-23T06:19:00.000Z"^^; - lc:departureDelay "360"^^; - lc:departureStop ; - lc:departureTime "2019-07-23T06:14:00.000Z"^^; - ; - "Ostende"; - ; - ; - . - a lc:Connection; - lc:arrivalDelay "180"^^; - lc:arrivalStop ; - lc:arrivalTime "2019-07-23T06:31:00.000Z"^^; - lc:departureDelay "180"^^; - lc:departureStop ; - lc:departureTime "2019-07-23T06:28:00.000Z"^^; - ; - "Bruges"; - ; - ; - . diff --git a/examples/paged-collection-with-order/second.ttl b/examples/paged-collection-with-order/second.ttl deleted file mode 100644 index 2f243de..0000000 --- a/examples/paged-collection-with-order/second.ttl +++ /dev/null @@ -1,44 +0,0 @@ -@prefix tree: . -@prefix hydra: . -@prefix dcterms: . -@prefix rdfs: . -@prefix schema: . -@prefix shacl: . -@prefix lc: . -@prefix xsd: . -@prefix void: . - - a hydra:Collection ; - rdfs:label "All arrivals and departures of Trains in Belgium"@en ; - void:subset ; #using the more general void:subset, as not the entire collection can be found from this node as there is no backlink - hydra:member , . - -## The data itself - - a lc:Connection; - lc:arrivalDelay "60"^^; - lc:arrivalStop ; - lc:arrivalTime "2019-07-23T06:36:00.000Z"^^; - lc:departureDelay "180"^^; - lc:departureStop ; - lc:departureTime "2019-07-23T06:32:00.000Z"^^; - ; - "Bruges"; - ; - ; - . - a lc:Connection; - lc:arrivalDelay "0"^^; - lc:arrivalStop ; - lc:arrivalTime "2019-07-23T06:44:00.000Z"^^; - lc:departureDelay "0"^^; - lc:departureStop ; - lc:departureTime "2019-07-23T06:36:00.000Z"^^; - ; - "Zottegem"; - ; - ; - . - - - diff --git a/examples/timeseries/14/8393/5467/2020-12-25T14%3A00%3A00.000Z.ttl b/examples/timeseries/14/8393/5467/2020-12-25T14%3A00%3A00.000Z.ttl deleted file mode 100644 index 5f6c274..0000000 --- a/examples/timeseries/14/8393/5467/2020-12-25T14%3A00%3A00.000Z.ttl +++ /dev/null @@ -1,67 +0,0 @@ -@prefix tree: . -@prefix hydra: . -@prefix dcterms: . -@prefix rdfs: . -@prefix lc: . -@prefix xsd: . -@prefix sosa: . -@prefix tiles: . -@prefix geo: . -@prefix cot: . -@prefix sh: . -@prefix geosparql: . -@prefix void: . - - a tree:Node ; - dcterms:isPartOf ; - # Historic data ends here - # Only a relation to go forward in time - tree:relation [ a tree:GreaterThanOrEqualToRelation ; - tree:node ; - tree:path sosa:resultTime ; - tree:value "2020-12-25T15:00:00.000Z"^^xsd:dateTime - ]; - tiles:longitudeTile "8393" ; - tiles:latitudeTile "5467"; - tiles:zoom "14"; - tree:timeQuery "2020-12-25T14:00:00.000Z"^^xsd:dateTime ; # the user followed the LowerThanRelation link from node to this node - tree:search _:b0 . - - - a tree:Collection ; - tree:view ; - void:subset ; - tree:member _:b1, _:b2, _:b3 . - -# Specific search form within this tile -_:b0 a hydra:IriTemplate ; - hydra:variableRepresentation hydra:BasicRepresentation ; - hydra:template "https://lodi.ilabt.imec.be/air/14/8393/5467/{t}"; - hydra:mapping [ a hydra:IriTemplateMapping ; - hydra:variable "t" ; - tree:path sosa:resultTime ; - hydra:property tree:timeQuery ; - hydra:required "false"^^xsd:boolean - ] . - -## The data itself -_:b1 a sosa:Observation; - sosa:resultTime "2020-12-25T14:05:00.000Z"^^xsd:dateTime ; - sosa:hasFeatureOfInterest <#Airquality/lat=51.23759466223419&long=4.415932092815638>, cot:AirQuality ; - sosa:observedProperty cot:PM1 . - -_:b2 a sosa:Observation ; - sosa:resultTime "2020-12-25T14:15:00.000Z"^^xsd:dateTime ; - sosa:hasFeatureOfInterest <#Airquality/lat=51.23759466223419&long=4.415932092815638>, cot:AirQuality ; - sosa:observedProperty cot:PM10 . - -_:b3 a sosa:Observation ; - sosa:resultTime "2020-12-25T14:55:00.000Z"^^xsd:dateTime ; - sosa:hasFeatureOfInterest <#Airquality/lat=51.23759466223419&long=4.415932092815638>, cot:AirQuality ; - sosa:observedProperty cot:NO2 . - -<#Airquality/lat=51.23759466223419&long=4.415932092815638> a sosa:FeatureOfInterest; - rdfs:label "Air quality at this point location"; - geosparql:asWKT "POINT (4.426460266113281 51.24171563651943)"^^geosparql:wktLiteral ; - geo:latitude "51.23759466223419"; - geo:longitude "4.415932092815638". \ No newline at end of file diff --git a/examples/timeseries/14/8393/5467/2020-12-25T15%3A00%3A00.000Z.ttl b/examples/timeseries/14/8393/5467/2020-12-25T15%3A00%3A00.000Z.ttl deleted file mode 100644 index c9f24dd..0000000 --- a/examples/timeseries/14/8393/5467/2020-12-25T15%3A00%3A00.000Z.ttl +++ /dev/null @@ -1,70 +0,0 @@ -@prefix tree: . -@prefix hydra: . -@prefix dcterms: . -@prefix rdfs: . -@prefix lc: . -@prefix xsd: . -@prefix sosa: . -@prefix tiles: . -@prefix geo: . -@prefix cot: . -@prefix sh: . -@prefix geosparql: . -@prefix void: . - - a tree:Node ; - dcterms:isPartOf ; - # Two relations allow to define a time interval - # tree:LessThanRelation will be mostly used to recursively go back in time - tree:relation [ a tree:LessThanRelation ; - tree:node ; - tree:path sosa:resultTime ; - tree:value "2020-12-25T15:00:00.000Z"^^xsd:dateTime ; - tree:remainingItems 3 - ], - [ - a tree:GreaterThanOrEqualToRelation ; - tree:node ; - tree:path sosa:resultTime ; - tree:value "2020-12-25T14:00:00.000Z"^^xsd:dateTime ; - tree:remainingItems 3 - ]; - tiles:longitudeTile "8393" ; - tiles:latitudeTile "5467"; - tiles:zoom "14"; - tree:timeQuery "2020-12-25T15:00:00.000Z"^^xsd:dateTime ; # the user requested results that occured after 2020-12-25T15:00:00.000Z - tree:search _:b0 . - - - a tree:Collection ; - tree:view ; - void:subset ; - tree:member _:b1, _:b2 . - -# Specific search form within this tile -_:b0 a hydra:IriTemplate ; - hydra:variableRepresentation hydra:BasicRepresentation ; - hydra:template "https://lodi.ilabt.imec.be/air/14/8393/5467/{t}"; - hydra:mapping [ a hydra:IriTemplateMapping ; - hydra:variable "t" ; - tree:path sosa:resultTime ; - hydra:property tree:timeQuery ; - hydra:required "false"^^xsd:boolean - ] . - -## The latest observations -_:b1 a sosa:Observation; - sosa:resultTime "2020-12-25T15:05:00.000Z"^^xsd:dateTime ; - sosa:hasFeatureOfInterest <#Airquality/lat=51.23759466223419&long=4.415932092815638>, cot:AirQuality ; - sosa:observedProperty cot:NO2 . - -_:b2 a sosa:Observation ; - sosa:resultTime "2020-12-25T15:10:00.000Z"^^xsd:dateTime ; - sosa:hasFeatureOfInterest <#Airquality/lat=51.23759466223419&long=4.415932092815638>, cot:AirQuality ; - sosa:observedProperty cot:PM10 . - -<#Airquality/lat=51.23759466223419&long=4.415932092815638> a sosa:FeatureOfInterest; - rdfs:label "Air quality at this point location"; - geosparql:asWKT "POINT (4.426460266113281 51.24171563651943)"^^geosparql:wktLiteral ; - geo:latitude "51.23759466223419"; - geo:longitude "4.415932092815638". diff --git a/examples/timeseries/airQualityObservations.ttl b/examples/timeseries/airQualityObservations.ttl deleted file mode 100644 index 4a0346f..0000000 --- a/examples/timeseries/airQualityObservations.ttl +++ /dev/null @@ -1,84 +0,0 @@ -@prefix tree: . -@prefix hydra: . -@prefix dcterms: . -@prefix rdfs: . -@prefix lc: . -@prefix xsd: . -@prefix sosa: . -@prefix tiles: . -@prefix geo: . -@prefix cot: . -@prefix sh: . -@prefix geosparql: . -@prefix void: . - - a tree:Node ; - # Let's specify the geospatial boundaries of the root node of a tile - # Option 1: use WKT literal - tree:relation [ - a tree:GeospatiallyContainsRelation ; - tree:path ( sosa:hasFeatureOfInterest geosparql:asWKT ); - tree:node ; - tree:value "POLYGON((4.41650390625 51.248163159055906, 4.41650390625 51.2344073516346, 4.4384765625 51.2344073516346, 4.4384765625 51.248163159055906, 4.41650390625 51.248163159055906))"^^geosparql:wktLiteral - ] ; - # Option 2: use lat long relations - tree:relation [ - a tree:GreaterThanOrEqualToRelation ; - tree:path ( sosa:hasFeatureOfInterest geo:longitude ); - tree:node ; - tree:value "4.41650390625" - ] ; - tree:relation [ - a tree:LessThanOrEqualToRelation ; - tree:path ( sosa:hasFeatureOfInterest geo:longitude ); - tree:node ; - tree:value "4.4384765625" - ] ; - tree:relation [ - a tree:LessThanOrEqualToRelation ; - tree:path ( sosa:hasFeatureOfInterest geo:latitude ); - tree:node ; - tree:value "51.248163159055906" - ] ; - tree:relation [ - a tree:GreaterThanOrEqualToRelation ; - tree:path ( sosa:hasFeatureOfInterest geo:latitude ); - tree:node ; - tree:value "51.2344073516346" - ] ; - tree:search _:b0 . - - a tree:Collection ; - rdfs:label "All airquality observations of our city."@en ; - # Here you can find the root node of the collection - tree:view ; - # (Optional) Point to the root node per tile - void:subset . # This will redirect to the page with the latest sensor data from that tile. In this example, this will redirect to - -_:b0 a hydra:IriTemplate ; - hydra:variableRepresentation hydra:BasicRepresentation ; - hydra:template "https://lodi.ilabt.imec.be/air/{z}/{x}/{y}/{t}"; - hydra:mapping [ - a hydra:IriTemplateMapping ; - hydra:variable "x" ; - hydra:property tiles:longitudeTile ; - hydra:required "true"^^xsd:boolean ; - sh:minInclusive "8393" ; - sh:maxInclusive "8393" - ], [ - a hydra:IriTemplateMapping ; - hydra:variable "y" ; - hydra:property tiles:latitudeTile ; - hydra:required "true"^^xsd:boolean ; - sh:hasValue "5467" - ], [ a hydra:IriTemplateMapping ; - hydra:variable "z" ; - hydra:property tiles:zoom ; - hydra:required "true"^^xsd:boolean ; - sh:hasValue "14" - ], [ a hydra:IriTemplateMapping ; - hydra:variable "t" ; - tree:path sosa:resultTime ; - hydra:property tree:timeQuery ; - hydra:required "false"^^xsd:boolean - ] . \ No newline at end of file diff --git a/spec.bs b/spec.bs deleted file mode 100644 index fa88f77..0000000 --- a/spec.bs +++ /dev/null @@ -1,27 +0,0 @@ - - -
path: specs/0-introduction.md
-
path: specs/1-discovery.md
-
path: specs/2-traversing.md
-
path: specs/3-search.md
-
path: specs/5-imports.md
-
path: specs/6-compatibility.md
- -
path: vocabulary.md
diff --git a/specs/0-introduction.md b/specs/0-introduction.md deleted file mode 100644 index 07b2335..0000000 --- a/specs/0-introduction.md +++ /dev/null @@ -1,112 +0,0 @@ -# Overview # {#overview} - -An overview of the TREE specification with the TREE collection, a reference to the first focus node of its members, and the relations to other nodes from the current node. - -The TREE specification introduces these core concepts: - * a tree:Collection is a subclass of dcat:Dataset ([[!vocab-dcat-3]]). The specialization being that it is a DCAT dataset a collection of members. It typically has these properties when described in a node: - - tree:member points at the first focus node from which to retrieve and extract all quads of a member. - - tree:view points to the current tree:Node you’re visiting. - - tree:shape indicates the [[!SHACL]] shape to which each member in the collection adheres. - - tree:viewDescription links to a description of the view (a tree:ViewDescription). Multiple descriptions MAY be provided that MUST be combined. - * a tree:Node: is a page on which relations to other pages are described through the tree:relation predicate, and/or through which a next tree:Node can be found by using the tree:search form. - * a tree:Relation is a relation from one node to another. An extension of this class indicates a specific type of relation (e.g., a tree:GreaterThanRelation). A relation typically has these properties: - - a tree:node the URL of the other node - - a tree:path indicating to which of the members' properties this relation applies - - a tree:value indicating a value constraint on the members' values - - a tree:remainingItems defining how many members can be reached when following this relation - * a tree:ViewDescription is a subclass of dcat:DataService and serves a tree:Collection. - - a tree:search describes a search form that allows an agent to jump to a specific tree:Node. - -The first step when creating a TREE hypermedia interface is defining a collection of members: - -
- ```turtle - ex:Collection1 a tree:Collection; - rdfs:label "A Collection of subjects"@en; - tree:member ex:Subject1, ex:Subject2 . - - ex:Subject1 a ex:Subject ; - rdfs:label "Subject 1" ; - ex:value 1 . - - ex:Subject2 a ex:Subject ; - rdfs:label "Subject 2" ; - ex:value 2 . - ``` -
- -From the moment this collection of members grows too large for one page, a fragmentation needs to be created in which an initial set of member can be found on an entry node, and more members can be found by interpreting the TREE hypermedia controls. This is illustrated by the next example: - -
- ```turtle - > HTTP GET https://example.org/Node1 - - ex:Collection1 a tree:Collection; - tree:view ex:Node1 ; - tree:member ex:Subject1, ex:Subject2 . - - ex:Node1 a tree:Node ; - tree:relation ex:R1,ex:R2 . - - ex:R1 a tree:GreaterThanOrEqualToRelation ; - tree:node ex:Node3 ; # This is the URL of another page - tree:value 3; - tree:path ex:value . - - ex:R1 a tree:LessThanRelation ; # This is very useful for a client that is looking for a value 10 or greater - tree:node ex:Node3 ; # This is the URL of another page - tree:value 10; - tree:remainingItems 7 ; - tree:path ex:value . - - ex:R2 a tree:GreaterThanOrEqualToRelation ; - tree:node ex:Node4 ; # This is the URL of another page - tree:value 10; - tree:remainingItems 10 ; - tree:path ex:value . - - ex:Subject1 a ex:Subject ; - rdfs:label "Subject 1" ; - ex:value 1 . - - ex:Subject2 a ex:Subject ; - rdfs:label "Subject 2" ; - ex:value 2 . - ``` -
- -
- Thanks to the [member extraction algorithm](#member-extraction-algorithm), a data publisher can choose to define their members in different ways: - 1. As in the examples above: all quads with the object of the tree:member quads as a subject (and recursively the quads of their blank nodes) are by default included (see also [[!CBD]]), except when they would explicitely not be included in case 3, when the shape would be closed. - 2. Out of band / in band: - - when no quads of a member have been found, the member will be dereferenced. This allows to publish the member on a separate page. - - part of the member can be maintained elsewhere when a shape is defined (see 3) - 3. By defining a more complex shape with tree:shape, also nested entities can be included in the member - 4. By putting the triples in a named graph of the object of tree:member, all these triples will be matched. -
- -# Definitions # {#formalizations} - -A tree:Collection is a set of tree:Members. The set of members MAY be empty. - -A tree:Member is a set of (at least one) quad(s) defined by the member extraction algorithm (next subsection). - -A tree:Node is a dereferenceable resource containing tree:Relations and a subset of () members of the collection. In a tree:Node, both the set of tree:Relations as the subset of members MAY be empty. The same member MAY be contained in multiple nodes. - -A tree:Relation is a function denoting a conditional link to another tree:Node. - -A tree:Node, apart from the root node, has exactly one other tree:Node linking into it through one or more relations. - -Note: The condition of multiple tree:Relations to the same tree:Node MUST be combined with a logical AND. - -A View is a specific set of interlinked tree:Nodes, that together contain all members in a collection. A specific view will adhere to a certain growth or tree balancing strategy. In one View, completeness MUST be guaranteed, unless the View has a retention policy which becomes possible in LDES. - -A tree:search form is an IRI template, that when filled out with the right parameters becomes a tree:Node IRI, or when dereferenced will redirect to a tree:Node from which all members in the collection that adhere to the described comparator can be found. - -## The member extraction algorithm ## {#member-extraction-algorithm} - -The set of quads the are part of the member, are defined by the [shape templates algorithm](https://w3id.org/tree/specification/shape-templates), provided as a separate report to this specification. -It is a combination of Concise Bounded Descriptions, named graphs and Shape Templates. -The latter uses the sh:NodeShape from the tree:shape property on the collections as an indication of the topology of the member graph. - -Note: The way we process SHACL shapes into Shape Template is important to understand in order to know when an HTTP request will be triggered when designing SHACL shapes. A cardinality constraint not being exactly matched or a sh:pattern not being respected will not trigger an HTTP request, and instead just add the invalid quads to the Member. This is a design choice: we only define triggers for HTTP request from the SHACL shape to come to a complete set of quads describing the member the data publisher pointed at using tree:member. diff --git a/specs/1-discovery.md b/specs/1-discovery.md deleted file mode 100644 index f6bbc05..0000000 --- a/specs/1-discovery.md +++ /dev/null @@ -1,59 +0,0 @@ -# Discovery and source selection# {#hypermedia} - -TREE tackles discovery and source selection on three levels: i) interface discovery, ii) view discovery, and iii) dataset discovery. -Interface discovery discovers what collection the current page is part of, and discovers what the next possible HTTP requests are through relations and search forms. -One dataset can have multiple views that can be published across different servers, selecting one for a certain use case is part of the view discovery. -Dataset discovery is then selecting a tree:Collection of interest. - - -## Interface discovery ## {#interface-discovery} - -Interface discovery starts when a URL is provided to a specific tree:Node. -A node from which all members of a collection can be discovered (an “entry node”), can be found through a triple stating ex:C1 tree:view ex:N1 with ex:C1 being a tree:Collection and ex:N1 being a tree:Node. - -When the current page is a tree:Node, there MUST be a property linking the current page URL to the URI of the tree:Collection. However, not from all tree:Nodes all members can be reached, and therefore 2 other properties can be used: void:subset, or the inverse property, dcterms:isPartOf. - -ex:C1 tree:view <> . links the current page to the tree:Collection. - -We refer to next chapters for traversing across multiple relations, or for using search forms. - -## View discovery ## {#multiple-views} - -Todo: This will be reworked - -Note: How a client picks the right view is use case specific. The tree:ViewDescription’s properties can help in that regards. - -In order to prioritize a specific view link, the relations and search forms in the entry nodes can be studied for their relation types, path or remaining items. -The class tree:ViewDescription indicates a specific TREE structure on a tree:Collection. -Through the property tree:viewDescription a tree:Node can link to an entity that describes the view, and can be reused in data portals as the dcat:DataService. - -
- ```turtle - ## What can be found in a tree:Node - ex:N1 a tree:Node ; - tree:viewDescription ex:View1 . - - ex:C1 a tree:Collection ; - tree:view ex:N1 . - - ## What can be found on a data portal - ex:C1 a dcat:Dataset . - ex:View1 a tree:ViewDescription, dcat:DataService ; - dcat:endpointURL ex:N1 ; # The entry point that can be advertised in a data portal - dcat:servesDataset ex:C1 . - ``` -
- -When there is no tree:viewDescription property in this page, a client either already discovered the description of this view in an earlier tree:Node, either the current tree:Node is implicitly the ViewDescription. Therefore, when the property path tree:view → tree:viewDescription does not yield a result, the view properties MUST be extracted from the object of the tree:view triple. -A tree:Node can also be double typed as the tree:ViewDescription. A client must thus check for ViewDescriptions on both the current node without the tree:viewDescription qualification, as on the current node with the tree:viewDescription link. - - -## Dataset discovery ## {#multiple-collections} - -When multiple collections are found by a client, it can choose to prune the collections based on the tree:shape property. -Therefore a data publisher SHOULD annotate a tree:Collection instance with a SHACL shape. -The tree:shape points to a SHACL description of the shape (sh:NodeShape). - -Note: the shape can be a blank node, or a named node on which you should follow your nose when it is defined at a different HTTP URL. - -Note: For compatibility with the [Solid specifications](https://solidproject.org/TR/), a ShEx shape may also be given (see the chapter on compatibility bellow). diff --git a/specs/2-traversing.md b/specs/2-traversing.md deleted file mode 100644 index 1238e84..0000000 --- a/specs/2-traversing.md +++ /dev/null @@ -1,76 +0,0 @@ -# The tree:Relations # {#relations} - -The initial configuration of the tree:Collection and the description of the view is always provided when the view has been discovered, either in a separate document describing the view, either in the entry node itself. The configuration MUST be reused on any subsequent tree:Node. - -While discovering and traversing the interface, a client MUST take the descriptions on top of the Node, the View and the Collection with it. - -## Traversing relations ## {#traversing} - -A tree:Node element MAY have one or more tree:relation properties. A relation is an entity of the type tree:Relation, and MAY have a more specific type. A tree:Relation MUST have one tree:node object of the type tree:Node. By default, all nodes need to be followed, unless the client is able to select this relation for pruning (see next section). - -The tree:Relation’s tree:value SHOULD be set. The object of tree:value SHOULD be accompanied by a data type when it is a literal value. - -Every tree:Relation SHOULD have a tree:path, indicating the path from the member to the object on which the tree:Relation applies. For the different ways to express or handle a tree:path, we refer to [2.3.1 in the shacl specification](https://www.w3.org/TR/shacl/#x2.3.1-shacl-property-paths). All possible combinations of e.g., shacl:alternativePath, shacl:inversePath or shacl:inLanguage in the SHACL spec can be used. When shacl:alternativePath is used, the order in the list will define the importance of the order when evaluating the tree:Relation. A wildcard in the path is limited to the tree:shape of the tree:Collection. -The result of the evaluation of the tree:path, is the value that must be compared to the tree:value. - -Every tree:Relation MAY provide a tree:remainingItems. A client MAY use tree:remainingItems to estimate the completeness of the downloaded elements to the end-user. - -Note: When traversing, a client SHOULD keep a list of already visited pages, as despite this being the TREE spec, circular references and back-links are not explicitly prohibited. - -A tree:import MAY be defined in the tree:Relation instance. When there is a tree:path defined, and when the relation is flagged interesting to follow, the import link needs to be downloaded in order to find the necessary literals to be compared (it is thus already a tree:ConditionalImport. - -Note: An example of a tree:import is given [in the repository](https://github.com/TREEcg/specification/blob/master/examples/geospatially-ordered-public-transport/first.ttl#L27). - -When dereferencing the object of a tree:node triple, the client MUST follow redirects. The URL to be used as the tree:Node URL is the last URL after redirects. - -Note: This enables rebalancing search trees. - -## Fallbacks ## {#fallbacks} - -When there is no tree:view triple provided, a client MUST use the tree:Collection from the previous page and still continue extracting members, and extract further relations defined on the current page URL. - -When there are no tree:members and/or no tree:Collection defined, then still a tree:Relation can be defined. The tree:path in the tree:Relation then refers to a pattern that can start from every triple in the page. - -When no tree:path is defined, the tree:value MUST be compared to all members’ triples that *can be compared to* the tree:value as defined by the type of the relation (or when no members or collection are defined, on every triple in the page). -When due to rdfs:range incompatibility, the object cannot be compared, the object will not be considered for comparison. - -Note: This may enable server developers to indicate an index on all literals of the members (e.g., a prefix relation on title, description and body text) without having to indicate all of the alternative paths in the tree:path. - -The target object of a tree:path SHOULD be materialized in the current Node document, but when it is not, the object MAY be considered implicit on the condition both tree:path and tree:member are defined. -In contrast to sh:path, a tree:path MAY refer to an implicit property and may not be materialized in the current response. This may break SPARQL processors that did not yet come across the object before in their query plan. However, the tree may still be useful for query processors that, for example, prioritize queries according to the user’s location, and first download nodes that are nearby the user. Therefore, the materialized location of the object is not needed. While not recommended, possible heuristics could try to infer the data, could try to fetch it through another tree:Collection, or retrieve it using URI dereferencing. - -## Specific relations ## {#relationsubclasses} - -When the *only* type given for a certain Relation is tree:Relation, then the client must dereference all of the nodes. While this may seem useless, it can be used for the same use case as a hydra:PartialCollectionView. - -For other types check the chapter on relation types in the vocabulary [](#Relation). - -### Comparing strings ### {#strings} - -String values have three specific type of relations: the tree:PrefixRelation, the tree:SubstringRelation and the tree:SuffixRelation. - -Note: We experimented with server-chosen locales such that ça suffit can also be found when following a tree:PrefixRelation with a tree:value "c" (which at this moment is not supported). That would require an understanding of locales, and [browser/JavaScript support for locales is too low to be useful at this point](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl#Locale_identification_and_negotiation). - -Also the comparator relations such as tree:GreaterThanRelation can be used. -The strings MUST then be compared according to *case sensitive unicode ordering*. - -When a tree:path is defined, mind that you also may have to check the language of the element using the property shacl:inLanguage -More languages MAY be set. -When no language is set, all strings are compared. - -Note: If you want to have one resource containing both e and é as a prefix, you will have to create multiple relations to the same tree:Node. - -### Comparing named nodes ### {#named-nodes} - -When using comparator relations such as tree:GreaterThanRelation, named nodes must be compared as defined in the [ORDER BY section of the SPARQL specification](https://www.w3.org/TR/sparql11-query/#modOrderBy). - -### Comparing geospatial features ### {#geospatial} - -The tree:GeospatiallyContainsRelation is the relation than can be used to express all further members will be contained within a geospatial region defined by the WKT String in the tree:value. - -When using tree:GeospatiallyContainsRelation, the tree:path MUST refer to a literal containing a WKT string, such as geosparql:asWKT. - -### Comparing time literals ### {#time} - -When using relations such as tree:LessThanRelation or tree:GreaterThanRelation, the time literals need to be compared according to these 3 possible data types: xsd:date, xsd:dateTime or xsd:dateTimeStamp. - diff --git a/specs/3-search.md b/specs/3-search.md deleted file mode 100644 index 2d7ea5a..0000000 --- a/specs/3-search.md +++ /dev/null @@ -1,83 +0,0 @@ -# Search forms # {#searching} - -Searching through a TREE will allow you to immediately jump to the right tree:Node. -TREE relies on the [Hydra search specification](http://www.hydra-cg.com/spec/latest/core/#hydra:search) for its search forms. -It does however extend Hydra with specific search properties (hydra:IriTemplate) for different types of search forms, and searches starting from a tree:ViewDescription, to which the search form is linked with tree:search. -The behaviour of the search form fully depends on the specific property, for which TREE introduces a couple of specific properties: - -## Geospatial XYZ tiles search form ## {#xyztiles} - -Three properties allow to specify a geospatial XYZ tiles template (also known as slippy maps). - 1. tree:longitudeTile describes the X value - 2. tree:latitudeTile descrbes the Y value - 3. tree:zoom describes the zoom level - -All properties expect positive integers. - -
- ```turtle - a tree:Collection ; - dcterms:title "A prototype tree:Collection for Linked OpenStreetMap’s roads"@en ; - tree:view . - - a tree:Node ; - tree:viewDescription . - - a tree:ViewDescription ; - tree:search [ - a hydra:IriTemplate ; - hydra:template "https://tiles.openplanner.team/planet/20201103-095900/{z}/{x}/{y}" ; - hydra:variableRepresentation hydra:BasicRepresentation ; - hydra:mapping [ - a hydra:IriTemplateMapping ; - hydra:variable "x"; - hydra:property tree:longitudeTile; - hydra:required true - ],[ - a hydra:IriTemplateMapping ; - hydra:variable "y"; - hydra:property tree:latitudeTile; - hydra:required true - ],[ - a hydra:IriTemplateMapping ; - hydra:variable "z"; - hydra:property tree:zoom; - hydra:required true - ] - ] . - - ``` -
- -This search form describes a specific search form that uses a quad tree. The zoom level describes the depth, the longitudeTile and latitudeTile describe the x and y index of the fragmentation. (e.g., on zoom level 0, there’s 1 tile, on zoom level 1, there are 4 tiles, etc.). - -## Searching through a list of objects ordered by time ## {#timesearch} - -Same as the previous example but with the predicate tree:timeQuery expecting an xsd:dateTime. -This time however, when the page itself does not exist, a redirect is doing to happen to the page containing the timestamp. -A tree:path can indicate the time predicate which is intended. - -
- ```turtle - a tree:Collection ; - dcterms:title "An example collection with a time search view"@en ; - tree:view . - - a tree:Node ; - tree:viewDescription . - - a tree:ViewDescription ; - tree:search [ - a hydra:IriTemplate ; - hydra:template "https://example.org/{generatedAt}" ; - hydra:variableRepresentation hydra:BasicRepresentation ; - hydra:mapping [ - a hydra:IriTemplateMapping ; - hydra:variable "generatedAt"; - tree:path prov:generatedAtTime; - hydra:property tree:timeQuery; - hydra:required true - ] - ] . - ``` -
diff --git a/tree.ttl b/tree.ttl index 7c82975..de2594d 100644 --- a/tree.ttl +++ b/tree.ttl @@ -23,15 +23,13 @@ tree: a foaf:Document ; foaf:primaryTopic tree:Ontology; cc:license ; - dct:creator , . + dct:creator . foaf:name "Pieter Colpaert"; foaf:mbox "pieter.colpaert@ugent.be". - foaf:name "Ruben Taelman" . - tree:Ontology a owl:Ontology ; rdfs:label "TREE"@en; - rdfs:comment "A hypermedia specification for fragmenting collections."@en. + rdfs:comment "A hypermedia specification for fragmenting collections of members."@en. ######## Classes @@ -40,15 +38,19 @@ tree:Collection a rdfs:Class ; rdfs:label "Collection"@en; rdfs:comment "A tree:Collection is a collection containing members. The members may be spread across multiple tree:Nodes."@en . -tree:ViewDescription a rdfs:Class ; - rdfs:subClassOf dcat:DataService ; - rdfs:label "View Description"@en ; - rdfs:comment "Describes a specific TREE structure on top of the tree:Collection"@en . +tree:SearchTree a rdfs:Class ; + rdfs:subClassOf dcat:Distribution ; + rdfs:label "Search Tree"@en; + rdfs:comment "A tree:SearchTree publishes the members of a collection."@en . tree:Node a rdfs:Class ; rdfs:label "Node"@en; rdfs:comment "A tree:Node is a node that may contain relations to other nodes."@en . +tree:RootNode a rdfs:Class ; + rdfs:label "Root Node"@en; + rdfs:comment "A tree:RootNode is the access point into a search tree."@en . + tree:Relation a rdfs:Class ; rdfs:label "Relation"@en ; rdfs:comment "A class describing the relation between two nodes"@en. @@ -96,7 +98,7 @@ tree:EqualToRelation a rdfs:Class ; tree:NotEqualToRelation a rdfs:Class ; rdfs:subClassOf tree:Relation ; - rdfs:label "Not equal To Relation"@en . + rdfs:label "Not Equal To Relation"@en . tree:GeospatiallyContainsRelation a rdfs:Class ; rdfs:subClassOf tree:Relation ; @@ -112,9 +114,9 @@ tree:InBetweenRelation a rdfs:Class ; tree:viewDescription a rdf:Property ; rdfs:label "View Description"; - rdfs:comment "Links together a tree:Node with its description of this TREE structure"@en; - rdfs:domain tree:Node ; - rdfs:range tree:ViewDescription . + rdfs:comment "Links together a tree:Node with a description of this search tree through the search tree itself or through a data service"@en; + rdfs:domain tree:Node . + #rdfs:range owl:oneOf ( tree:SearchTree dcat:DataService ) . # Just leaving the range open for now tree:relation a rdf:Property ; rdfs:label "Relation"@en; @@ -134,6 +136,13 @@ tree:node a rdf:Property ; rdfs:domain tree:Relation; rdfs:range tree:Node. +tree:rootNode a rdf:Property ; + rdfs:subPropertyOf dcat:accessURL ; + rdfs:label "Has Root Node"@en; + rdfs:comment "A view has a root node that can be used to start traversing the search tree"@en; + rdfs:domain tree:View ; + rdfs:range tree:RootNode . + tree:value a rdf:Property ; rdfs:label "Value"@en ; rdfs:comment "The value the node linked in the node relation is compared to"@en . @@ -159,7 +168,6 @@ tree:member a rdf:Property ; tree:search a rdf:Property ; rdfs:label "Search"@en ; rdfs:comment "The Node can be searched for child nodes."@en ; - rdfs:domain tree:Node ; rdfs:range hydra:IriTemplate . tree:shape a rdf:Property ; diff --git a/vocabulary.md b/vocabulary.md index b1b1beb..5fcb739 100644 --- a/vocabulary.md +++ b/vocabulary.md @@ -44,10 +44,6 @@ The tree:Relation has specific sub-classes that implement a more sp A class to import a file or a stream based on a tree:path of properties. This way it can import the necessary data for complying to the SHACL shape, or evaluating a relation type. -### tree:ViewDescription ### {#ViewDescription} - -Describes a specific TREE structure on top of the tree:Collection. - ## Properties ## {#properties} ### tree:relation ### {#relation} @@ -151,8 +147,6 @@ A search form parameter: accompagnied by a tree:path, it indicates ### tree:viewDescription ### {#viewDescription} -Links together a tree:Node with its description of this TREE structure. +Links together any HTTP response with a view description on which things like retention policies, contact information of a server, etc. can be found. **Domain**: tree:Node - -**Range**: tree:ViewDescription