SHACL focus node & source Shape #88 #160

ensaremirerol · 2023-12-01T15:08:38Z

Basically added a extra flag for CLI tool named -addBlankNodes. With this flag enabled tool will append all blank nodes that references shapes file.

Here is the data/shape file I used in testing and the validation report outputted from tool with mention flag enabled

Data and Shape

@prefix : <https://biomedit.ch/rdf/sphn-schema/sphn#> .
@prefix chop: <https://biomedit.ch/rdf/sphn-resource/chop/> .
@prefix constraints: <https://biomedit.ch/rdf/sphn-schema-constraints/sphn#> .
@prefix dcterms: <http://purl.org/dc/terms/> .
@prefix eco: <http://purl.obolibrary.org/obo/ECO_> .
@prefix edam: <http://edamontology.org/> .
@prefix efo: <http://www.ebi.ac.uk/efo/EFO_> .
@prefix emdn: <https://biomedit.ch/rdf/sphn-resource/emdn/> .
@prefix genepio: <http://purl.obolibrary.org/obo/GENEPIO_> .
@prefix geno: <http://purl.obolibrary.org/obo/GENO_> .
@prefix icd-10-gm: <https://biomedit.ch/rdf/sphn-resource/icd-10-gm/> .
@prefix obi: <http://purl.obolibrary.org/obo/OBI_> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix skos: <http://www.w3.org/2004/02/skos/core#> .
@prefix snomed: <http://snomed.info/id/> .
@prefix so: <http://purl.obolibrary.org/obo/SO_> .
@prefix sphn-atc: <https://biomedit.ch/rdf/sphn-resource/atc/> .
@prefix sphn-loinc: <https://biomedit.ch/rdf/sphn-resource/loinc/> .
@prefix ucum: <https://biomedit.ch/rdf/sphn-resource/ucum/> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix ex: <http://example.org/> .

constraints:AccessDevice a sh:NodeShape ;
    sh:closed false ;
    sh:ignoredProperties ( rdf:type ) ;
    sh:property [ sh:maxCount 1 ;
            sh:message "constraint:AccessDevice_1",
                "message:MinCount, Or, or MaxCount constraint violated, expected minCount is 0 and maxCount is 1 and expected one of the following: https://biomedit.ch/rdf/sphn-schema/sphn#Code, http://snomed.info/id/105789008, https://biomedit.ch/rdf/sphn-schema/sphn#Terminology",
                "sparql:SELECT ?this (COUNT(?o) AS ?actualCount) ?actual WHERE { ?this <https://biomedit.ch/rdf/sphn-schema/sphn#hasTypeCode> ?o . ?o rdf:type ?actual . }",
                "template:MinCount, Or, or MaxCount constraint violated on {this} on hasTypeCode, expected minCount is 0 and maxCount is 1 and expected one of the following: https://biomedit.ch/rdf/sphn-schema/sphn#Code, http://snomed.info/id/105789008, https://biomedit.ch/rdf/sphn-schema/sphn#Terminology, found {actualCount} objects of type {actual}" ;
            sh:minCount 0 ;
            sh:or ( [ sh:class :Code ] [ sh:class snomed:105789008 ] [ sh:class :Terminology ] ) ;
            sh:path :hasTypeCode ],
        [ sh:maxCount 1 ;
            sh:message "constraint:AccessDevice_2",
                "message:MinCount, Or, or MaxCount constraint violated, expected minCount is 0 and maxCount is 1 and expected one of the following: https://biomedit.ch/rdf/sphn-schema/sphn#Code, https://biomedit.ch/rdf/sphn-schema/sphn#Terminology",
                "sparql:SELECT ?this (COUNT(?o) AS ?actualCount) ?actual WHERE { ?this <https://biomedit.ch/rdf/sphn-schema/sphn#hasProductCode> ?o . ?o rdf:type ?actual . }",
                "template:MinCount, Or, or MaxCount constraint violated on {this} on hasProductCode, expected minCount is 0 and maxCount is 1 and expected one of the following: https://biomedit.ch/rdf/sphn-schema/sphn#Code, https://biomedit.ch/rdf/sphn-schema/sphn#Terminology, found {actualCount} objects of type {actual}" ;
            sh:minCount 0 ;
            sh:or ( [ sh:class :Code ] [ sh:class :Terminology ] ) ;
            sh:path :hasProductCode ] ;
    sh:targetClass :AccessDevice .

ex:test a :AccessDevice ;
    :hasTypeCode ex:dummy ;
    :hasTypeCode ex:dummy2 .

ex:dummy a ex:dummyType1 .
ex:dummy2 a ex:dummyType2 .

Validation Report with Blank Node/s

@prefix :            <https://biomedit.ch/rdf/sphn-schema/sphn#> .
@prefix chop:        <https://biomedit.ch/rdf/sphn-resource/chop/> .
@prefix constraints: <https://biomedit.ch/rdf/sphn-schema-constraints/sphn#> .
@prefix dash:        <http://datashapes.org/dash#> .
@prefix dcterms:     <http://purl.org/dc/terms/> .
@prefix eco:         <http://purl.obolibrary.org/obo/ECO_> .
@prefix edam:        <http://edamontology.org/> .
@prefix efo:         <http://www.ebi.ac.uk/efo/EFO_> .
@prefix emdn:        <https://biomedit.ch/rdf/sphn-resource/emdn/> .
@prefix ex:          <http://example.org/> .
@prefix genepio:     <http://purl.obolibrary.org/obo/GENEPIO_> .
@prefix geno:        <http://purl.obolibrary.org/obo/GENO_> .
@prefix graphql:     <http://datashapes.org/graphql#> .
@prefix icd-10-gm:   <https://biomedit.ch/rdf/sphn-resource/icd-10-gm/> .
@prefix obi:         <http://purl.obolibrary.org/obo/OBI_> .
@prefix owl:         <http://www.w3.org/2002/07/owl#> .
@prefix rdf:         <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs:        <http://www.w3.org/2000/01/rdf-schema#> .
@prefix sh:          <http://www.w3.org/ns/shacl#> .
@prefix skos:        <http://www.w3.org/2004/02/skos/core#> .
@prefix snomed:      <http://snomed.info/id/> .
@prefix so:          <http://purl.obolibrary.org/obo/SO_> .
@prefix sphn-atc:    <https://biomedit.ch/rdf/sphn-resource/atc/> .
@prefix sphn-loinc:  <https://biomedit.ch/rdf/sphn-resource/loinc/> .
@prefix swa:         <http://topbraid.org/swa#> .
@prefix tosh:        <http://topbraid.org/tosh#> .
@prefix ucum:        <https://biomedit.ch/rdf/sphn-resource/ucum/> .
@prefix xsd:         <http://www.w3.org/2001/XMLSchema#> .

_:b0    sh:maxCount  1 ;
        sh:message   "template:MinCount, Or, or MaxCount constraint violated on {this} on hasTypeCode, expected minCount is 0 and maxCount is 1 and expected one of the following: https://biomedit.ch/rdf/sphn-schema/sphn#Code, http://snomed.info/id/105789008, https://biomedit.ch/rdf/sphn-schema/sphn#Terminology, found {actualCount} objects of type {actual}" , "sparql:SELECT ?this (COUNT(?o) AS ?actualCount) ?actual WHERE { ?this <https://biomedit.ch/rdf/sphn-schema/sphn#hasTypeCode> ?o . ?o rdf:type ?actual . }" , "message:MinCount, Or, or MaxCount constraint violated, expected minCount is 0 and maxCount is 1 and expected one of the following: https://biomedit.ch/rdf/sphn-schema/sphn#Code, http://snomed.info/id/105789008, https://biomedit.ch/rdf/sphn-schema/sphn#Terminology" , "constraint:AccessDevice_1" ;
        sh:minCount  0 ;
        sh:or        ( [ sh:class  :Code ]
                       [ sh:class  snomed:105789008 ]
                       [ sh:class  :Terminology ]
                     ) ;
        sh:path      :hasTypeCode .

[ rdf:type     sh:ValidationReport ;
  sh:conforms  false ;
  sh:result    [ rdf:type                      sh:ValidationResult ;
                 sh:focusNode                  ex:test ;
                 sh:resultMessage              "constraint:AccessDevice_1" , "sparql:SELECT ?this (COUNT(?o) AS ?actualCount) ?actual WHERE { ?this <https://biomedit.ch/rdf/sphn-schema/sphn#hasTypeCode> ?o . ?o rdf:type ?actual . }" , "message:MinCount, Or, or MaxCount constraint violated, expected minCount is 0 and maxCount is 1 and expected one of the following: https://biomedit.ch/rdf/sphn-schema/sphn#Code, http://snomed.info/id/105789008, https://biomedit.ch/rdf/sphn-schema/sphn#Terminology" , "template:MinCount, Or, or MaxCount constraint violated on {this} on hasTypeCode, expected minCount is 0 and maxCount is 1 and expected one of the following: https://biomedit.ch/rdf/sphn-schema/sphn#Code, http://snomed.info/id/105789008, https://biomedit.ch/rdf/sphn-schema/sphn#Terminology, found {actualCount} objects of type {actual}" ;
                 sh:resultPath                 :hasTypeCode ;
                 sh:resultSeverity             sh:Violation ;
                 sh:sourceConstraintComponent  sh:MaxCountConstraintComponent ;
                 sh:sourceShape                _:b0
               ] ;
  sh:result    [ rdf:type                      sh:ValidationResult ;
                 sh:focusNode                  ex:test ;
                 sh:resultMessage              "sparql:SELECT ?this (COUNT(?o) AS ?actualCount) ?actual WHERE { ?this <https://biomedit.ch/rdf/sphn-schema/sphn#hasTypeCode> ?o . ?o rdf:type ?actual . }" , "constraint:AccessDevice_1" , "template:MinCount, Or, or MaxCount constraint violated on {this} on hasTypeCode, expected minCount is 0 and maxCount is 1 and expected one of the following: https://biomedit.ch/rdf/sphn-schema/sphn#Code, http://snomed.info/id/105789008, https://biomedit.ch/rdf/sphn-schema/sphn#Terminology, found {actualCount} objects of type {actual}" , "message:MinCount, Or, or MaxCount constraint violated, expected minCount is 0 and maxCount is 1 and expected one of the following: https://biomedit.ch/rdf/sphn-schema/sphn#Code, http://snomed.info/id/105789008, https://biomedit.ch/rdf/sphn-schema/sphn#Terminology" ;
                 sh:resultPath                 :hasTypeCode ;
                 sh:resultSeverity             sh:Violation ;
                 sh:sourceConstraintComponent  sh:OrConstraintComponent ;
                 sh:sourceShape                _:b0 ;
                 sh:value                      ex:dummy
               ] ;
  sh:result    [ rdf:type                      sh:ValidationResult ;
                 sh:focusNode                  ex:test ;
                 sh:resultMessage              "template:MinCount, Or, or MaxCount constraint violated on {this} on hasTypeCode, expected minCount is 0 and maxCount is 1 and expected one of the following: https://biomedit.ch/rdf/sphn-schema/sphn#Code, http://snomed.info/id/105789008, https://biomedit.ch/rdf/sphn-schema/sphn#Terminology, found {actualCount} objects of type {actual}" , "message:MinCount, Or, or MaxCount constraint violated, expected minCount is 0 and maxCount is 1 and expected one of the following: https://biomedit.ch/rdf/sphn-schema/sphn#Code, http://snomed.info/id/105789008, https://biomedit.ch/rdf/sphn-schema/sphn#Terminology" , "constraint:AccessDevice_1" , "sparql:SELECT ?this (COUNT(?o) AS ?actualCount) ?actual WHERE { ?this <https://biomedit.ch/rdf/sphn-schema/sphn#hasTypeCode> ?o . ?o rdf:type ?actual . }" ;
                 sh:resultPath                 :hasTypeCode ;
                 sh:resultSeverity             sh:Violation ;
                 sh:sourceConstraintComponent  sh:OrConstraintComponent ;
                 sh:sourceShape                _:b0 ;
                 sh:value                      ex:dummy2
               ]
] .

HolgerKnublauch

Nice work overall! Certainly the last main class needs to be removed, but you may want to look into my other comments too before submitting an update?

HolgerKnublauch · 2023-12-01T15:16:16Z

src/main/java/org/topbraid/shacl/tools/BlankNodeFinder.java

+
+        Model blankNodes = ModelFactory.createDefaultModel();
+
+        Stream<Statement> statements = report.listStatements(null, SH.result, (RDFNode) null).toList().stream();


Just a minor comment here: this first creates a List and then a Stream. This may not scale well for large reports. Better would be to use an ExtendedIterator instead, which also has a map function

Changed it all list creation to use only iterators

HolgerKnublauch · 2023-12-01T15:18:23Z

src/main/java/org/topbraid/shacl/tools/BlankNodeFinder.java

+        statements.forEach(statement -> {
+            blankNodes.add(statement);
+            if (statement.getObject().isAnon()) {
+                blankNodes.addAll(resolveAllBlankNodes(model, statement.getResource()));


This can theoretically run into a recursion if the bnodes have cycles. That is extremely unlikely but is usually handled by passing around a Set of the reached subjects.

Also, it can theoretically cause a stackoverflow when you traverse deep rdf:Lists. Breadth-first traversal avoids that.

Please review current state of the code. I don't know if this is what you wanted

src/test/java/org/topbraid/shacl/FindBlankNodes.java

ensaremirerol · 2023-12-01T16:11:18Z

I've refactored code based on your feedback.
Thanks!

ensaremirerol added 3 commits December 1, 2023 15:50

Added ability to add blank nodes at the end of the report for CLI tool

5bc8c40

Added ability to add blank nodes at the end of the report for CLI tool

80f5795

Reverted back to old method of writing out the reports

1dafef0

HolgerKnublauch requested changes Dec 1, 2023

View reviewed changes

Optimized the code based on feedback

2651e0a

HolgerKnublauch approved these changes Dec 1, 2023

View reviewed changes

HolgerKnublauch merged commit c48ad9d into TopQuadrant:master Dec 1, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SHACL focus node & source Shape #88 #160

SHACL focus node & source Shape #88 #160

ensaremirerol commented Dec 1, 2023

HolgerKnublauch left a comment

HolgerKnublauch Dec 1, 2023

ensaremirerol Dec 1, 2023

HolgerKnublauch Dec 1, 2023

ensaremirerol Dec 1, 2023

ensaremirerol commented Dec 1, 2023


		Model blankNodes = ModelFactory.createDefaultModel();

		Stream<Statement> statements = report.listStatements(null, SH.result, (RDFNode) null).toList().stream();

SHACL focus node & source Shape #88 #160

SHACL focus node & source Shape #88 #160

Conversation

ensaremirerol commented Dec 1, 2023

HolgerKnublauch left a comment

Choose a reason for hiding this comment

HolgerKnublauch Dec 1, 2023

Choose a reason for hiding this comment

ensaremirerol Dec 1, 2023

Choose a reason for hiding this comment

HolgerKnublauch Dec 1, 2023

Choose a reason for hiding this comment

ensaremirerol Dec 1, 2023

Choose a reason for hiding this comment

ensaremirerol commented Dec 1, 2023