Update to new edition of minimum elements (2024)

[bact]

• The 3rd edition of "Framing Software Component Transparency" document is published on October 2024.
  https://www.cisa.gov/resources-tools/resources/framing-software-component-transparency-2024
• We may like to update the conformance checker to aligned with the new edition

[jspeed-meyers]

Thank you for pointing this out, @bact! I defer to @goneall and @kestewart. Do you want this tool to reflect the revised criteria in the new document cited above?

If so, that could be a good project for a Google Summer of Code intern.

[bact]

Yes, I think that will be useful and help adoption of SPDX.
Although I'm not certain about the scope of SPDX versions we would like to support.

Currently, ntia-conformance-checker supports SPDX 2.3.
To have the support for the Third edition of minimum elements in SPDX 2.3 is one thing.
To have the support for the Third edition of minimum elements also in SPDX 3.0 is another thing.

Considering the experimental support of SPDX 3.0 in spdx-tools dependency, an SPDX 2.3 is probably more feasible in the near term. Although the additional SPDX 3.0 support will do more favour for 3.0 adoption.

--

There is also a similar document from the German Federal Office for Information Security (BSI) that we could consider together:

• Technical Guideline TR-03183: Cyber Resilience Requirements for Manufacturers and Products Part 2: Software Bill of Materials (SBOM)

(and the Summer of Code is interesting)

[jspeed-meyers]

To be clear, I support updating the ntia-conformance-checker to implement the new edition of the minimum elements:

• if (A) Gary and Kate support it
• if (B) there is someone or some group of people that can make the necessary changes to the code while ensuring the test suite is also updated properly.

I can't currently justify B personally (at least as a volunteer maintainer), though I would be glad to supervise a GSOC intern or outside contributor to do it, reviewing PRs as necessary and providing advice.

[bact]

Thank you. Having played around a bit with tools-python SPDX model, I'm happy to look into this.

Let's see how Gary and Kate think about this.

[goneall]

I personally support updating the minimum elements. For compatibility, I would suggest that we have a command line option to select which version of the minimum elements is used - the original NTIA or the 3rd edition of the framing document.

[jspeed-meyers]

@bact or others: I'm glad to advise on how to implement this change. I suggest creating a design document first (perhaps a comment-only Google document or something similar) since there is potentially a lot to unpack and then arriving at a rough consensus before sending any PRs in.

[bact]

Thanks. I will try to come up with the design doc, maybe next week.

[jspeed-meyers]

Sounds good, @bact. And LMK if you want to brainstorm or if I can help in any way.

[bact]

@jspeed-meyers I have put some notes here: https://docs.google.com/document/d/1pueRxlxoM9n1eG9g6AihjLvybEBTd77m22mRYBQltpg/edit?usp=sharing

Note that sbomqs already supports FSCT v3 on SPDX 2.x, so may need to see what is still missing.

[jspeed-meyers]

@bact: Nice document.

IIUC, you are proposing to expand this tool's mandate from checking the conformance of an SPDX SBOM with the NTIA minimum elements to checking SPDX SBOM conformance with a range of frameworks. Do I understand your intention correctly?

In general, I want this tool be broadly useful. So I support this motion in the abstract.

I do have two concerns. First, would any users besides yourself find this useful? I worry about expanding the mandate of the tool without clear and strong evidence that MANY users would find this helpful. Second, who will do the creation and maintenance of such a tool? I have become a co-maintainer of this tool (and the company where I work USED to use this tool internally), but there are no longer clear incentives for me to do anything other than basic maintenance. I worry about adding lots of functionality, which will inevitably have bugs, and there being no set of maintainers able to debug and propose fixes.

Anyways, nice document! I support the idea in general.

For me, the most important opinions are those of @goneall and @kestewart. I defer to them.

[bact]

Thank you. Very useful comments.

1. On the demand side

Frankly I don't know. I believe there will be an increasing demand due to regulations and business needs, counting on the existence of a tool like sbomqs.

But of course, as currently sbomqs support more standards and formats (SPDX 2 and CycloneDX), there's no good reason to switch to the tool in this repo.

Unless there's a featue that other tools don't have yet (and that feature is essential enough, which I don't know what it is).

Personally, I want to use this tool to check the conformance of an SBOM against requirements in EU AI Act. This will link to the supply side, the creation and maintenance in the next point.

2. On the supply side

I am willing to create some of these features.

My idea is to start somewhere with less moving parts, to understand how things work, so standard documents like NTIA and FSCTv3 came to my mind. They are quite established and I don't have to worry much about interpretation.

After I understand how to technically check SBOM conformance, I will then continue to apply it with EU AI Act requirements.

I have an incentive for the creation because it will help with my study at university. But of course, one can question the maintenance in the long run after I left university (which personally I hope happens soon).

Maybe to ease the concern, each addition of standard support should be developed in a way that detached from the main program. A feature should be removed easily when required (the feature is no longer maintained and has bug/dependencies that will affect the main program, or the standard document is deprecated/revoked).

[jspeed-meyers]

But of course, as currently sbomqs support more standards and formats (SPDX 2 and CycloneDX), there's no good reason to switch to the tool in this repo.

Yeah, I think sbomqs is a perfectly good tool. The advantage, IMO, of this tool, ntia-conformance-checker, is not technical but organizational: it is housed within the spdx GitHub organization as an officially-supported implementation of NTIA conformance checking for SPDX SBOM documents. But if a user prefers sbomqs, then go for it!

Maybe to ease the concern, each addition of standard support should be developed in a way that detached from the main program.

I do think this would ease my concerns.

And, again, I'm glad to review PRs.

I still have the broader question though of: should this tool include SBOM quality standards beyond the NTIA minimum elements? I'm open to it. But I really do see this as a Gary and Kate question.

Also, I too originally got interested in this tool because of research (see here), so I understand your situation :)

[goneall]

I have a couple of opinions on this topic:

• This tool is quite valuable in that it is available through the online tools making it quite easy for those in the SPDX community to check conformance.
• I foresee a lot of demand for the 3rd edition framing document conformance checking at the minimum level - the other standards suggested would be nice to have as well, but I have a feeling most people will just be checking for the minimum
• To keep the scope reasonable, I would focus on only well specified published standard (which the proposal seems to do). Implementing a more general "quality" analysis would be quite a bit of work and seems to be well covered by the sbomqs work.

[jspeed-meyers]

@bact: Given @goneall's support, I would suggest, at least in the short term, a PR that focuses on the 3rd edition framing document conformance checking at the minimum level, implemented via a new option like the design document above. If you aren't totally opposed to more design document work, I would suggest sketching out what new functions need to be added to implement the 3rd edition framing document conformance checking. Once you and I have a rough consensus there, you can send in a PR.

And everything else can wait until a later day.

Does that satisfy your immediate research needs?

[bact]

@goneall @jspeed-meyers Thank you. Yes, I think it it sound to start from the "Minimum Expected" level of the FSCT v3.

Will sketching what needs to be added first as John suggested.

[bact]

@jspeed-meyers I have updated the sketch. Please look at the "Current design" and "New design proposals" sections. Thanks.

[jspeed-meyers]

I appreciate the sketch. I'm starting to understand the architecture. I asked some questions. I can tell you've thought about this more than I have. I admire your ability to create architectural options and describe them. Please review my questions at your convenience. Once you've answered them--and I suspect we're close to consensus--then I welcome a PR. Nice job.

[bact]

@jspeed-meyers Thanks a lot. Those questions made me think more. I have answered all (I think) in the Google Docs, some as comments, some as the "Example of Approach" sections. Please see if there's anything we can have more details.

[jspeed-meyers]

@bact: I think this design document answers all of my questions and explains your intent. Please proceed!

One note: I will be on paternity leave from the Wednesday of this week until January 6, 2025. I am afraid I will hardly be at my computer. @goneall can, of course, review PRs too, as his time allows. Anyways, I wanted you to know so that you didn't think I was avoiding any PRs. But when I return, I'm glad to help with PR reviews too.

[bact]

Oh. Don't worry. Spend the most out of your important break. Take care + congrats 🎉

[bact]

#224 is ready for initial (re)structural review before I will proceed into the compliance check implementation in more details.

[jspeed-meyers]

Looking good to me so far! Nice!