NIST Policy on the use of the name OSCAL in software, usnistgov/oscal-cli maintenance, and other topics

[aj-stein-gsa]

@iMichaela, these questions are related to #2080, but a different topic. Both #2084 (comment) and #2084 (comment) are not longer immediately relevant to the review of PR, so I will post it separately here for clarification.

So now we can address the questions, comments, and concerns that are not related to the PR below.

This is not something NIST can accept as proposed, despite its great intent of enhancing OSCAL testing methodology. Changes are needed.

Major ask: remove/rename the oscal npm package https://www.npmjs.com/package/oscal wrongly and confusingly named after the OSCAL (language) - a set of schemas. This is unacceptable to NIST, as documented in the issue GSA/oscal-js#4 - an issue already closed by @aj-stein-gsa which stated that oscal is used by the community in different situations and no confusion was created. I find no substance and no proof provided by this simple opinionated statement (aka "no confusion is created") .

The only claim of confusion thus far was in GSA/oscal-js#4. There is a claim about major confusion, but no proof provided, in the issue or elsewhere. Has anyone else formally communicated this confusion? Can you share it? I think it is safe for me to say "a comment says there is confusion, but nothing else found, I do not see any confusion."

To wit, I can think of many, many libraries that share a name with a data format, language, and/or process they support.

https://docs.python.org/3/library/json.html
https://docs.python.org/3/library/html.html
https://docs.oracle.com/en/java/javase/11/docs/api/java.xml/module-summary.html

Who is NIST in the claim "This is unacceptable to NIST ...": the OSCAL Team, you as OSCAL director, CSD leadership, or ITL leadership?

NIST is not endorsing the use on oscal name for the OSCAL schema validation because coming from NIST will cause confusion.

What does NIST endorsement mean? Who in NIST determines this endorsement (the OSCAL Team, you as OSCAL director, CSD leadership, or ITL leadership?)?

The name OSCAL is used by NIST, GSA, and many commercial and non-commercial organizations (open-source software, educational materials not commercially licensed) use the name "OSCAL" in domains (oscal.io; oscal.club are obvious examples), code, documentation, and other artifacts not made by NIST. Do we presume they represent OSCAL the schema in every occurence of the name?

Otherwise, are you implying that there is some branding, trademark, copyright or intellectual property policy that is published or will be published?

I also recall Dave's opinion aligned with mine since both of us fought for years community members' misunderstanding that OSCAL is a tool they can install. Providing an oscal tool that can be installed but which is not OSCAL the language (the set of schemas) will bring back the confusion, especially when coming from NIST.

If @david-waltermire feels strongly, I am confident he will chime in.

I will explain what I had not before regarding the nature of modern supply chain attacks is that to change the name opens the oscal package name in NPMJS repository to typo squatting attacks, and it is an increasingly common problem.

Not only that, NPMJS has documented policy and procedures because this approach of reusing known package names from past trusted maintainers is an increasingly common attack vector. So having an alternate name does not mean we can stop using that name.

We can consider the package name and scope issue, but that is not a change that can be done immediately within the context of this review for that PR. Also note, moving to the new name does not mean we can abandon the old one for the reasons described above. I hope that explains the reason in detail not provided earlier.

Totally agree with @wendellpiez general direction. However, this PR proposes to 'augment' OSCAL schemas validation using the harness FedRAMP is using. NIST OSCAL Makefile already includes validation with usnistgov\oscal-cli but suffers when usnistgov\oscal-cli is not updated to the latest OSCAL version.

So can we confirm if usnistgov/oscal-cli being maintained?

A chicken and an egg problem which will not be addressed by this PR. This PR is not proposing a tool for others to use, but rather an additional way for using an oscal-cli which is not usnistgov\oscal-cli + OSCAL Server developed by GSA for their purpose.

GSA staff contribute to the metaschema-framework of oscal-cli, but it is not owned by GSA, or it would be in the GSA GitHub organization.

There have been a variety of worthwhile improvements to proposed changes to the Metaschema specification and the supporting tooling, including CLI. If you would like me or other community members of the metaschema-framework community to present on them, let us know. Many of them will benefit the larger OSCAL community, not just users who happen to be GSA developers.

[iMichaela]

It was me and Dave that for years (many) had to explain over and over to interested parties that OSCAL is not a tool that can be installed and used to automate your compliance ... I am shocked that after being part of the NIST OSCAL team and you still do not understand this major concern. It is true you served as technical director for a very short time and maybe did not have the chance to interact with the OSCAL newbies.

Regarding all changes made by GSA team to the forked usnistgov/oscal-cli under a non-governmental repository, aka metaschema-framework, NIST management directions is to accept PR submitted to the origin (NIST) that make sense to all community members, but to not use those forks. The proposal that metaschema-framework becomes the owner of the metaschema and oscal-cli was turned down and communicated by the CSD Division Chief in writing to involved parties. My responsibility is to implement it.

[aj-stein-gsa]

It was me and Dave that for years (many) had to explain over and over to interested parties that OSCAL is not a tool that can be installed and used to automate your compliance

I am going to skip responding to personal opinions about my tenure at NIST and keep it objective. Let's focus on this point.

There is a specific and important difference between "OSCAL can't automate all compliance" and "we should discourage or prohibit naming a software package with the name OSCAL because its main purpose is to process that data format but that will confuse people on the capabilities of the OSCAL information model and data formats" and they are not equivalent concepts. I have never in any experience met people that have sustained confusion if they ask this once and get an answer because they are just encountering something for the first time.

For example, this perspective is akin to saying that developers or users cannot talk about Excel or name library that processes Excel because they will confuse it with the notion of Excel data files. So I am not sure how to address this concern barring any specific ongoing problem with it.

Regarding all changes made by GSA team to the forked usnistgov/oscal-cli under a non-governmental repository, aka metaschema-framework, NIST management directions is to accept PR submitted to the origin (NIST) that make sense to all community members, but to not use those forks. The proposal that metaschema-framework to become the owners of the metaschema and oscal-cli was turned down and communicated in writing by the CSD Division Chief in writing to involved parties. My responsibility is to implement it.

Who in NIST management, the division chief? Where was this written? I am familiar with the proposal that I proposed, but a meeting was cancelled and no written record was provided and as far as I know no decision was made.

I will presume the response means the maintenance of the usnistgov version of oscal-cli is undetermined at this time.

If you are interested in the Metaschema specification changes and CLI fork and how it can benefit the OSCAL community, let me know. We are more than willing to brief on that. Perhaps that can inform future decisions about alignment with the usnistgov versions if they're maintained.

[david-waltermire]

No blame is being assigned here. The situation is that these NIST repositories are not being maintained and haven't been for a significant amount of time, while the metaschema-framework versions are actively maintained. The question is what to do now?

Why not use the open source metaschema-framework tools instead that are actively maintained at a production level?

[aj-stein-gsa]

Just to set the record straight for all:

1. the metaschema, metaschema-java, liboscal-java, and oscal-cli were NOT maintained by OSCAL team (me included). ... Everything not done prior to AJ's departure from NIST (fall 2024) is NIST's fault BUT not OSCAL team's fault.

I completely agree with Dave's comment: I do not think anyone is at fault or anyone wants to assign blame. Again, whether or not I fully agree with this characterization and summary of events, I am trying to focus objectively on problems we share and possible solutions. I am talking about present and future. I am discussing how, as a community member, to support everyone, not just FedRAMP and our use cases.

I simply asked: is the CLI maintained? Where is the NIST position or policy you summarized on not using any other version, maintained or otherwise, if written? If there is nothing published, oh well, that is fine. I just wanted clarification. The initial summary of the position was much more strict and specific (in #2080 you implied there were policies prohibiting use of any non-NIST schemas, maybe software as well) than the concluding summary in your previous comment ("I received regarding the maintenance of those repos, was to do my best and work with the community's support"). Regarding this specific concern.

validation with usnistgov\oscal-cli but suffers when usnistgov\oscal-cli is not updated to the latest OSCAL version.

I specifically am talking to you here, in this discussion post, to understand if there is a way forward to help with a community-maintained tooling that fixes bugs and makes the CLI tooling more reliable. This thread originated in a PR. I believe there are updated comments about how using the usnistgov/oscal-cli variant has errors that are based in bugs since fixed in the metaschema-framework fork.

Dave and I are asking: is it not in the interest of the community to use that tooling, temporarily or in the long-term, to improve analysis of the Metaschema models (it has come a long way in 5+ years, the last year with significant improvement per Dave's metrics)?

2. The guidance I received regarding the maintenance of those repos, was to do my best and work with the community's support until we replace the vacant positions,

So, I am a community member, am attempting to offer such community support, as are Brian, Dave, and Paul. In light of vacant positions, what can we do to work together?

I will repeat my offer: in some form, do you want to know about the proposed Metaschema specific enhancements and CLI tooling improvements from our community-based projects?

As for the loops around the other topic, I will presume the lack of answers about copyright/trademark/IP policy is OSCAL (the specification, the documentation, and code) is still and will remain under a public domain license, per 15 USC Section 105. I am very supportive of that and think that serves the community, but you have implied that NIST has some additional position or policy re community efforts that may dissuade other community members unless NIST "endorses" their project 100%.

I wanted to give you the opportunity to say that is not the case as to not dissuade further engagement.

[iMichaela]

Why not use the open source metaschema-framework tools instead that are actively maintained at a production level?

I already provided the answer to this question. But I will repeat is here since it might have gotten lost:
Regarding all changes made by GSA team to the forked usnistgov/oscal-cli under a non-governmental repository, aka metaschema-framework, NIST management's direction is to accept PRs submitted to the origin (NIST's repo) that make sense to all community members, and to not use those forks not managed by NIST. The proposal that metaschema-framework becomes the owner of the metaschema and oscal-cli was turned down and communicated by the CSD's Division Chief in writing to involved parties. My responsibility is to implement the resolution my management decided.

The reason for the resolution was alluded in the conversation above, but I will re-iterate it here. NIST needs to have full controls over all OSCAL-related repos so they can be used, as needed, to generate the OSCAL package that will constitute the PWI (proposed work item) submitted to the international standards body, when OSCAL maturity is reached and time comes to submit it. If this will not be possible, the submission will be reduced to a simple OSCAL XML set of schema and all JSON users will be affected. This last scenario is not something I support, and hence my fight for implementing management's directions in a way that includes and supports all users.

There was a time (fall 2023- summer 2024) when I was not an admin for those repos but there was a NISTers admin team and PRs, a time when the metaschema-framework repo could submit PRs against the usnistgov repos, and keep in metaschema-framework the FedRAMP specific features. It is regrettable, but history cannot be changed, and instead of pointing fingers I would rather prefer for us and the community members to work together to remediate it. Now that I have admin rights for all those repositories, we can work together (if there si interest), and update the usnistgov/oscal-cli and related repos, with the features the OSCAL users (ALL not just the USG ones) could benefit from. We have CNCF collaborators (e.g the teams led by @eddie-knight and @ancatri) interested in reviewing and supporting this efforts as well.

[aj-stein-gsa]

I already provided the answer to this question. But I will repeat is here since it might have gotten lost: ...

It must not have been clear the multiple times I asked, but I asked who made this policy and where they documented it, not that the policy exists because you already summarized it:

Who in NIST management, the division chief? Where was this written? I am familiar with the proposal that I proposed, but a meeting was cancelled and no written record was provided and as far as I know no decision was made.

At this point I will take your answers to be: we have a policy that we have summarized publicly for you, but we cannot clarify whether or not it is written or who authored and approved it. That's fine, I just think it should be transparent to community members.

The reason for the resolution was alluded in the conversation above, but I will re-iterate it here. NIST needs to have full controls over all OSCAL-related repos so they can be used, as needed, to generate the OSCAL package that will constitute the PWI (proposed work item) submitted to the international standards body, when OSCAL maturity is reached and time comes to submit it. If this will not be possible, the submission will be reduced to a simple OSCAL XML set of schema and all JSON users will be affected. This last scenario is not something I support, and hence my fight for implementing management's directions in a way that includes and supports all users.

This is not material to the question I asked, but it clarifies a lot of my concerns re the theme of my questions: does NIST run a community-driven project with public domain code or do they believe they unilaterally control and selectively choose what to "endorse" or not? It seems the latter.

And if "full control" means you must have full control of all software dependencies used (which continues to be implied by lack of clear answers in this thread), with or without community support, you cannot realistically conform to this policy now or in the long-term. The majority of code and data to develop and maintain OSCAL models and documentation are not developed or maintained by NIST staff. And that is not to be a critique or insult: that is a healthy, wonderful by-product of all our projects and most of the open-source world is the same, as to not reinvent wheels. If you want concrete evidence, we or any other community member can provide that as it pertains to OSCAL.

But to most clear, the answer to this underlying question demonstrates NIST's current perspective is hostile to all other individuals and organizations in the community, not just A.J. the person or FedRAMP. My perception of this philosophy underpinning the policy is why I created the issue and wanted to make clear by posing these questions, so that separate of a PR other community members see this perspective from NIST made explicit.

a time when the metaschema-framework repo could submit PRs against the usnistgov repos

This time transpired this way for an obvious, practical reason: many PRs, of which Dave provided some examples above, were not reviewed or merged. WIll the outstanding PRs get reviewed by you or community members referenced below? Starting at that time, the metaschema-framework versions were created to address blocking bug and add feature fixes everyone needs. To that end, we implemented non-trivial changes to allow co-existence of both implementations. Then and now, we cannot simply send PRs with our release branches to usnistgov/{metaschema-java,liboscal-java,oscal-cli} as a result and you can just merge it. If so, we want to support this community, we would have done so.

keep in metaschema-framework the FedRAMP specific features.

The metaschema-framework version of the tools do not have FedRAMP specific features. This confusion relates to a question I had previously asked and went unanswered twice.

I will repeat my offer: in some form, do you want to know about the proposed Metaschema specific enhancements and CLI tooling improvements from our community-based projects?

You have not taken this offer yet, so I will presume the answer is no, you are not interested, and I will stop asking. If you want to revisit that topic, feel free. If you want to review the bug fixes and feature enhancements, that are not FedRAMP specific, to understand the benefit to the community, I offered you the opportunity for that express purpose. Beyond that, I am not going to address subjective impressions of the metaschema-framework versions of the tool. We can present objectively what the code does and does not do, and the code can also speak for itself re implementation.

It is regrettable, but history cannot be changed, and instead of pointing fingers I would rather prefer for us and the community members to work together to remediate it. Now that I have admin rights for all those repositories, we can work together (if there si interest), and update the usnistgov/oscal-cli and related repos, with the features the OSCAL users (ALL not just the USG ones) could benefit from. We have CNCF collaborators (e.g the teams led by @eddie-knight and @ancatri) interested in reviewing and supporting this efforts as well.

You cannot simply update those repos. If you want further details, feel free to reach out. I am always happy to work with members of the community, but there is no trivial way to make the usnistgov versions reach parity, even if you assert the summary of the policy above. That said, if you want to know more detail as to why, and find some approach with a group of community members, let me know how you would like to proceed.

I am going to mark this as resolved for now, because we are going in circles and not clarifying the ambiguity around policy practice that is hostile to the community. I can accept it and brief FedRAMP leadership to determine next steps.

[iMichaela]

NIST is certainly not hostile! Your comments are!
Forking NIST projects, further developing them under a repo owned by a private person and not by a USG agency, and then demanding for NIST to use those forks is what has not been approved by the CSD division chief. The request to simply give those NIST repos to what you call 'community project' was discussed with the ITL Deputy Director and CSD Division Chief and the decision was communicated to people involved. Not being included in the email that communicated the decision despite the fact that you were a NISTer at that time, was management's decision. Your current direct lead was. I communicated the decision and I hope you will accept it, and I will continue to serve my agency to the best of my abilities and implement my management's decision. Your aggressive messages are not going to change this decision.

[eddie-knight]

Hey folks, can I recommend that this conversation is taken offline?

[aj-stein-gsa]

NIST is certainly not hostile! Your comments are! Forking NIST projects, further developing them under a repo owned by a private person and not by a USG agency, and then demanding for NIST to use those forks is what has not been approved by the CSD division chief.

I don't really understand the dramatic tone. I said the policy is hostile, I didn't say NIST was. The majority of software that constitutes OSCAL code and documentation is freely licensed or open source licensed software maintained by individuals. The same is true of all OSCAL related projects that are free or open source licensed. It is troubling that NIST staff provide specifications and code in the public domain, then criticize the very use that public domain license facilitates. We too provide code in the public domain and encourage reuse without conditions, that's the point of the license and a requirement for federal employees under US Code. I would expect that to be clearly understood. You are free to use or not use specific community tools, I asked you to clarify why not the metaschema-framework tools I help maintain. The application of your policy is inconsistent, and that's ok, but I accepted my non-answer because it seems the motivation is not appropriate to public discussion, and I am happy to accept that. Generally, it discourages others too, not just me. If you, be it Michaela, the OSCAL Team, CSD, ITL, or NIST must "endorse" projects or code on a case by case basis, it is hostile to adoption, even if there is sufficient bandwidth for NIST staff to do so. All I hoped for in this thread is you would edit the guidance from Matt Scholl and publish it as a bonafide policy in the repository, wiki, or project websites. I am not going to go continue with summary of history past and grievances, you'll notice I politely ignore that every time. I am ready to help discuss what you would want to do if you or your colleagues want to create PRs to the currently unmaintained usnistgov versions. It's not relevant here but it is a significant effort that will require significant coordination. If you are interested, just email me.

Hey folks, can I recommend that this conversation is taken offline?

I appreciate the sentiment, but I believe in the transparency of open source project is not just "the code," but also topics of governance and disagreement, whether it is comfortable or not for me. I hope you can respect that, but I am done after this post anyway. In the past key decision points were not published or not kept current around design, implementation, governance, etc. This made my previous role as a NIST staff member difficult, and I am trying to benefit the community with transparency around critical topics that since remained undocumented. I do not have to agree with every decision or policy, but they should be transparent, no?