README.md

Dataset: dbpedia-live (development version)

DBpedia Live was a real-time service that monitored edits on Wikipedia and published a stream of changes to the DBpedia knowledge graph. This dataset contains only the "added" triples in the stream, so it does not include deletes or other types of changes. Only one month (January 2014) is covered at the moment, but the dataset can be easily expanded in the future (the service stopped functioning in 2021). The stream's elements are irregular in size, depending on the volume of traffic on Wikipedia at a given moment and how the DBpedia Live service was able to cope with it. See also the paper.

The dataset was extensively cleaned to fix or remove bad IRIs, bad Unicode, and invalid literals.

This README is a snapshot of documentation for the latest development version of the dataset. Full documentation for all versions can be found on the website.

General information

• Title: DBpedia Live (en)
• Identifier: dbpedia-live
• Has version: dev
• Theme:
  • Encyclopaedia (eurovoc:4137)
  • Open data (eurovoc:c_5ea6e5c4)
• Creator:
  • Wikipedia contributors (1)
    • Name: Wikipedia contributors
    • Homepage: https://www.wikipedia.org/
  • Piotr Sowiński (2)
    • Name: Piotr Sowiński
    • Comment: Processing the dataset
    • Nickname: Ostrzyciel
    • Homepage:
      • (https://orcid.org/0000-0002-2543-9461)
      • Ostrzyciel (https://github.com/Ostrzyciel)
  • DBpedia Association (3)
    • Name: DBpedia Association
    • Homepage: https://www.dbpedia.org/contact/
• License: https://spdx.org/licenses/CC-BY-SA-3.0
• Rights: The dataset contains attributions of the original source of the data (links to Wikipedia). (en)
• Source:
  • https://doi.org/10.1108/00330331211221828
  • https://www.dbpedia.org/resources/live/
• Date Issued: 2023-05-09
• Date Modified: 2024-08-29
• Landing page: dbpedia-live (dev)
• Conforms To: Metadata (https://w3id.org/riverbench/schema/metadata)

Technical metadata

• Has stream type usage:
  • RDF stream type usage (1)
    • Type: RDF stream type usage (stax:RdfStreamTypeUsage)
    • Comment: The dataset can be viewed as a stream of graphs corresponding to batches of updates of Wikipedia articles. (en)
    • Has stream type: RDF graph stream (stax:graphStream)
  • RDF stream type usage (2)
    • Type: RDF stream type usage (stax:RdfStreamTypeUsage)
    • Comment: The dataset can be viewed as a flattened stream of triples. (en)
    • Has stream type: Flat RDF triple stream (stax:flatTripleStream)
• Has stream element count: 166,204
• Has stream element split:
  • Type: Stream elements split by time (rb:TimeStreamElementSplit)
  • Comment: Each element corresponds to a batch of recent changes from Wikipedia. The size of the batch may have been influenced by the traffic on Wikipedia, the load on the system, and other factors, so the element sizes are irregular. (en)
  • Has temporal property: http://dbpedia.org/ontology/wikiPageExtracted
• Uses vocabulary: http://dbpedia.org/ontology/
• Conforms to W3C RDF 1.1 specification: yes
• Conforms to W3C RDF-star draft specification as of December 17, 2021: yes
• Uses generalized triples: no
• Uses generalized RDF datasets: no
• Uses RDF-star: no

Distributions

Full stream distribution

• Title: Full stream distribution
• Identifier: stream-full
• Has file name: stream_full.tar.gz
• Has stream type usage:
  • Type: RDF stream type usage (stax:RdfStreamTypeUsage)
  • Comment: The dataset can be viewed as a stream of graphs corresponding to batches of updates of Wikipedia articles. (en)
  • Has stream type: RDF graph stream (stax:graphStream)
• Has distribution type:
  • Full distribution (rb:fullDistribution)
  • Stream distribution (rb:streamDistribution)
• Has stream element count: 166,204
• Byte size: 256.4 MB
• Media type: text/turtle
• Packaging format: application/tar
• Compression format: application/gzip
• Download URL: https://w3id.org/riverbench/datasets/dbpedia-live/dev/files/stream_full.tar.gz

Full Jelly distribution

• Title: Full Jelly distribution
• Identifier: jelly-full
• Has file name: jelly_full.jelly.gz
• Has stream type usage:
  • RDF stream type usage (1)
    • Type: RDF stream type usage (stax:RdfStreamTypeUsage)
    • Comment: The dataset can be viewed as a flattened stream of triples. (en)
    • Has stream type: Flat RDF triple stream (stax:flatTripleStream)
  • RDF stream type usage (2)
    • Type: RDF stream type usage (stax:RdfStreamTypeUsage)
    • Comment: The dataset can be viewed as a stream of graphs corresponding to batches of updates of Wikipedia articles. (en)
    • Has stream type: RDF graph stream (stax:graphStream)
• Has distribution type:
  • Full distribution (rb:fullDistribution)
  • Jelly distribution (rb:jellyDistribution)
• Has stream element count: 166,204
• Byte size: 351.2 MB
• Media type: application/x-jelly-rdf
• Compression format: application/gzip
• Download URL: https://w3id.org/riverbench/datasets/dbpedia-live/dev/files/jelly_full.jelly.gz

Full flat distribution

• Title: Full flat distribution
• Identifier: flat-full
• Has file name: flat_full.nt.gz
• Has stream type usage:
  • Type: RDF stream type usage (stax:RdfStreamTypeUsage)
  • Comment: The dataset can be viewed as a flattened stream of triples. (en)
  • Has stream type: Flat RDF triple stream (stax:flatTripleStream)
• Has distribution type:
  • Flat distribution (rb:flatDistribution)
  • Full distribution (rb:fullDistribution)
• Has stream element count: 166,204
• Byte size: 282.2 MB
• Media type: application/n-triples
• Compression format: application/gzip
• Download URL: https://w3id.org/riverbench/datasets/dbpedia-live/dev/files/flat_full.nt.gz

100K elements stream distribution

• Title: 100K elements stream distribution
• Identifier: stream-100k
• Has file name: stream_100K.tar.gz
• Has stream type usage:
  • Type: RDF stream type usage (stax:RdfStreamTypeUsage)
  • Comment: The dataset can be viewed as a stream of graphs corresponding to batches of updates of Wikipedia articles. (en)
  • Has stream type: RDF graph stream (stax:graphStream)
• Has distribution type:
  • Partial distribution (rb:partialDistribution)
  • Stream distribution (rb:streamDistribution)
• Has stream element count: 100,000
• Byte size: 209.1 MB
• Media type: text/turtle
• Packaging format: application/tar
• Compression format: application/gzip
• Download URL: https://w3id.org/riverbench/datasets/dbpedia-live/dev/files/stream_100K.tar.gz

100K elements Jelly distribution

• Title: 100K elements Jelly distribution
• Identifier: jelly-100k
• Has file name: jelly_100K.jelly.gz
• Has stream type usage:
  • RDF stream type usage (1)
    • Type: RDF stream type usage (stax:RdfStreamTypeUsage)
    • Comment: The dataset can be viewed as a stream of graphs corresponding to batches of updates of Wikipedia articles. (en)
    • Has stream type: RDF graph stream (stax:graphStream)
  • RDF stream type usage (2)
    • Type: RDF stream type usage (stax:RdfStreamTypeUsage)
    • Comment: The dataset can be viewed as a flattened stream of triples. (en)
    • Has stream type: Flat RDF triple stream (stax:flatTripleStream)
• Has distribution type:
  • Jelly distribution (rb:jellyDistribution)
  • Partial distribution (rb:partialDistribution)
• Has stream element count: 100,000
• Byte size: 288.6 MB
• Media type: application/x-jelly-rdf
• Compression format: application/gzip
• Download URL: https://w3id.org/riverbench/datasets/dbpedia-live/dev/files/jelly_100K.jelly.gz

100K elements flat distribution

• Title: 100K elements flat distribution
• Identifier: flat-100k
• Has file name: flat_100K.nt.gz
• Has stream type usage:
  • Type: RDF stream type usage (stax:RdfStreamTypeUsage)
  • Comment: The dataset can be viewed as a flattened stream of triples. (en)
  • Has stream type: Flat RDF triple stream (stax:flatTripleStream)
• Has distribution type:
  • Flat distribution (rb:flatDistribution)
  • Partial distribution (rb:partialDistribution)
• Has stream element count: 100,000
• Byte size: 230.1 MB
• Media type: application/n-triples
• Compression format: application/gzip
• Download URL: https://w3id.org/riverbench/datasets/dbpedia-live/dev/files/flat_100K.nt.gz

10K elements stream distribution

• Title: 10K elements stream distribution
• Identifier: stream-10k
• Has file name: stream_10K.tar.gz
• Has stream type usage:
  • Type: RDF stream type usage (stax:RdfStreamTypeUsage)
  • Comment: The dataset can be viewed as a stream of graphs corresponding to batches of updates of Wikipedia articles. (en)
  • Has stream type: RDF graph stream (stax:graphStream)
• Has distribution type:
  • Partial distribution (rb:partialDistribution)
  • Stream distribution (rb:streamDistribution)
• Has stream element count: 10,000
• Byte size: 66.7 MB
• Media type: text/turtle
• Packaging format: application/tar
• Compression format: application/gzip
• Download URL: https://w3id.org/riverbench/datasets/dbpedia-live/dev/files/stream_10K.tar.gz

10K elements Jelly distribution

• Title: 10K elements Jelly distribution
• Identifier: jelly-10k
• Has file name: jelly_10K.jelly.gz
• Has stream type usage:
  • RDF stream type usage (1)
    • Type: RDF stream type usage (stax:RdfStreamTypeUsage)
    • Comment: The dataset can be viewed as a stream of graphs corresponding to batches of updates of Wikipedia articles. (en)
    • Has stream type: RDF graph stream (stax:graphStream)
  • RDF stream type usage (2)
    • Type: RDF stream type usage (stax:RdfStreamTypeUsage)
    • Comment: The dataset can be viewed as a flattened stream of triples. (en)
    • Has stream type: Flat RDF triple stream (stax:flatTripleStream)
• Has distribution type:
  • Jelly distribution (rb:jellyDistribution)
  • Partial distribution (rb:partialDistribution)
• Has stream element count: 10,000
• Byte size: 91.2 MB
• Media type: application/x-jelly-rdf
• Compression format: application/gzip
• Download URL: https://w3id.org/riverbench/datasets/dbpedia-live/dev/files/jelly_10K.jelly.gz

10K elements flat distribution

• Title: 10K elements flat distribution
• Identifier: flat-10k
• Has file name: flat_10K.nt.gz
• Has stream type usage:
  • Type: RDF stream type usage (stax:RdfStreamTypeUsage)
  • Comment: The dataset can be viewed as a flattened stream of triples. (en)
  • Has stream type: Flat RDF triple stream (stax:flatTripleStream)
• Has distribution type:
  • Flat distribution (rb:flatDistribution)
  • Partial distribution (rb:partialDistribution)
• Has stream element count: 10,000
• Byte size: 73.7 MB
• Media type: application/n-triples
• Compression format: application/gzip
• Download URL: https://w3id.org/riverbench/datasets/dbpedia-live/dev/files/flat_10K.nt.gz