-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Updates to Testing ontology #774
Conversation
Also includes proposed changes; will be removed when proposal resolved.
f5ce40a
to
32177be
Compare
32177be
to
a3a6c7d
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should add more ontology to allow the capture of higher and lower fidelity trace. For the boeing over lay this would mean there is a SBVT_TestProcedure -verifies-> {SRS_Req orSubDD} and IDD_Test -verifies->Signal. More generally this should be in the core ontology as TEST_PROCEDURE -verifies-> ENITITY, TEST_STEP -verifies-> ENTITY, and TEST -verifies-> ENTITIY. A similar structure is need for the log/results side
Boeing-Ontology/ontology/Boeing.sadl
Outdated
IDD_Doc is a type of DOCUMENT. | ||
|
||
// subclass from core ontology related to SBVT and IDD | ||
SBVT_Test_Procedure is a type of TEST_PROCEDURE. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should add more ontology to allow the capture of higher and lower fidelity trace. For the boeing over lay this would mean there is a SBVT_TestProcedure -verifies-> {SRS_Req orSubDD} and IDD_Test -verifies->Signal. More generally this should be in the core ontology as TEST_PROCEDURE -verifies-> ENITITY, TEST_STEP -verifies-> ENTITY, and TEST -verifies-> ENTITIY. A similar structure is need for the log/results side
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This PR does have a TEST--verifies-->ENTITY association at present, so that
concern is covered.
A TEST_STEP is really just the identification of a TEST, along with any
subsequent TEST_STEPs, so the TEST_STEP doesn't add any verification that isn't
already covered by the TEST it identifies and therefore shouldn't need a
"verifies" property.
The TEST_PROCEDURE has no additional context other than being a COLLECTION of
initial TEST_STEP entries, so the only additional verification claim it could
make would be relative to the aggregation of the TESTs. However, adding a
"verifies" property at the TEST_PROCEDURE level raises a concern of coherence: if
a TEST_PROCEDURE were to indicate that it verifies a high-level ENTITY but the
set of TEST entries underneath that TEST_PROCEDURE are not an exact correlation
to all of the sub-ENTITY elements of that ENTITY, which one is correct (the
TEST_PROCEDURE.verifies or the individual TEST.verifies)? This also affects
extensibility: if a TA1 determines that an additional test is needed to validate
the higher-level construct, then the TEST_PROCEDURE can no longer be said to
verify that construct. In this consideration, the intent is that in our ontology
the TEST object is the atomic unit of verification and that any implication that
higher-level elements are fully verified is a TA3 evaluation/conclusion that we
don't have enough information to supply or synthesize.
In general, the heirarchy of TEST_PROCEDURE::TEST_STEP::TEST is concerned with
describing the relationship of each TEST to any other TEST, rather than an
association of different testing importances, but there is no constraint on the
fidelity of a particular TEST, so it should be possible to define a TEST to
associate to any desired target ENTITY to capture the desired fidelity.
Also includes proposed changes; will be removed when proposal resolved.
a3a6c7d
to
cc193c0
Compare
@tuxji any idea why this CI failed? |
I'm not certain if this is the cause, but I see the following two messages which probably indicate that shellcheck did not pass:
I noticed that the TESTING_updates_kwq branch is 5 commits ahead, 116 commits behind master. Your best bet is to rebase the TESTING_updates_kwq branch on master (there shouldn't be any conflicts), force push, and see if CI passes: git switch TESTING_updates_kwq
git pull
git rebase origin/master
git push --force-with-lease |
@tuxji OK, I was confused because this PR didn't touch any shell files but I think what happened is shellcheck got updated and master accomodated that after this branch, so I've merged with master and that fixed the linting problem, but now there's a docker image CI problem that I'm unsure about. |
I reran the CI job and it got past that problem, so your change didn't cause that problem (probably a slow computer or network caused it). You can merge the PR anyway, @kquick |
Thanks, @tuxji. That CI job is actually not completing and shows having run for 18h40m and counting, but I think we can chalk that up to gremlins rather than bugs. This is ready for merging after discussion and approval in the data model decision team, @kityansiu . |
…CK into TESTING_updates_kwq
@kquick : the data model decision team will make a decision to merge after v12 |
The TESTING ontology works to capture a broad range of testing considerations; not all of those considerations are apparent or available when ingesting data, so the scenarios now describe some of the fallback representations when less data is available.
f8d8882
to
01dfe62
Compare
Graphical representation of the new TESTING ontology included: https://github.com/ge-high-assurance/RACK/blob/TESTING_updates_kwq/RACK-Ontology/Graphs/TESTING_scoped.svg |
f2f0ef9
to
515c51e
Compare
515c51e
to
535413b
Compare
Fully approved by DMDT. |
See RACK_Ontology/Orienteering.md for a background and description of TESTING changes.