✨ Handle arbitrary metrics order and add testing

[superbuggy]

Changes:

• Handles any ordering of CVSS metrics (is this desirable?)
• Defines test suite
• Defines continuous integration workflow

[pandatix]

Arbitrary metrics order is invalid according to spec...

Furthermore, the current approach within this SIG for testing is developing an assessment framework so developers can easily handle this part. This PR goes against this effort by bypassing it.
The CVSS v4 calculator serves as a reference for other librarires maintainers, but implementing 500k lines of code just for testing is not acceptable to me.

[skontar]

@pandatix OK, I understand that arbitrary metric order is not correct, that makes sense.

However, I do not understand how adding tests is a problem and "bypassing" any other effort. If this is a reference implementation, why having tests which pass and which can be re-used by others, is wrong? Please elaborate. Also, it is not 500k lines of code, but of test vectors.

I am completely fine removing it from this project, just please give a more detailed rationale. Thanks!

[pandatix]

Hey, sorry for the late answer everyone.

Adding tests is not directly "bypassing" the SIG efforts, it is a good thing actually. From an external point of view (like an implementation maintainer/developer) the RedHat implementation is the de facto source of trust. NIST used it for their own calculator, many others accoring to our analysis.

This PR comes in parallel of this effort: one of the requirements we drew is to provide this test suite in an efficient manner (e.g., a file served by a CDN, a versionned file on GitHub, a GH release asset) and with a limited set of test cases (under 10k if possible) that would cover most cases without needing exhaustivity. Another one is to provide both valid and invalid test cases.
Indeed, we cannot expect an implementation to handle 5GB of data to say that we are confident enough, especially when the implementation is not performant (CI would take minutes up to hours to run at every commit, quickly becoming a burden for maintainers).

To avoid having parallel efforts, I think we should wait for the official FIRST.ORG SIG CVSS test data rather than creating one, propagating it to the community, then having to reach out maintainers to update their work one more time (update source of trust, consumption strategy, data model). We should hop on coordination rather than individual efforts bringing noise to the effort of CVSS v4 adoption.

[skontar]

@pandatix I see. So if I understand correctly, the best course of action is to wait for official FIRST.ORG SIG CVSS test vectors, right?

[pandatix]

Yes, I think so 😄