Skip to content

Latest commit

 

History

History
122 lines (105 loc) · 9.55 KB

tooling.md

File metadata and controls

122 lines (105 loc) · 9.55 KB

Tooling metadata

The tooling.json metadata file contains:

  1. the tools property, where the tools are described, and
  2. the criteria, having the mapping between the SQAaaS criteria1 and the tools.

Tool properties

Tools are categorized under the (programming) language they apply to, so e.g. Python's bandit tool shall be added under the Python key:

"tools": {
    "Python": {
        "bandit": {
            "docs": "https://bandit.readthedocs.io/",
            "docker": {
                "image": "secfigo/bandit",
                "reviewed": "1970-01-01"
            }
        }
    }
}

NOTE: the name of the languages are compliant with the ones used in the GitHub's linguist tool. The reason lies in the fact that file extensions for any given language are obtained from linguist metadata.

The following table summarizes the properties that ought to be set in the tool definition:

Tool property Type Description Required
docs string (url) URL to the tool's official documentation ✔️
docker object See Docker section ✔️
executable string Name of the executable. The tool's name is used by default.
This is only required when the executable to be used is different from the tool's name
template string Name of the template to use to compose the final command (not a concatenation of args))
args object See Arguments section
reporting object See Reporting section

Docker (docker property)

The docker property includes the information related to the availability of the Docker image that contains the tool. Hence, it offers two main ways of getting the Docker image, either 1) pulling an existing image from a Docker registry, or 2) build the image from a Dockerfile. One of the two properties must be defined:

Property Type Description Required
image string (url) Docker registry URL (defaults to Docker Hub) ✅ (only if dockerfile is not defined
dockerfile string (path) Relative path to the Dockerfile. This file shall be maintained in the present repository, under the criterion folder it applies to ✅ (only if image is not defined
reviewed string (date) Date the image was last used (format YYYY-MM-DD) ✔️
oneshot boolean Set this value to False if you don't want the SQAaaS API to handle it as a oneshot image (i.e. adding a sleep command, default: True)

Arguments (args property)

The args property enables the definition of the arguments involved in the tool execution. The type of argument can fall into the three categories set out below:

  • subcommand: many tools break up their functionality into subcommands. One popular example is the git tool that provides multiple subcommands (e.g. git add, git commit, ..).
  • positional: those arguments that are required and that are defined only by their value. They can be used both with a command or a subcommand. Continuing with the example above, the git add subcommand always require a positional argument (e.g. git add file1).
  • optional: those arguments that might be provided, but they are not required. The option name, which contains a single dash for the short version and two dashes for the long version, can be used both in conjunction with a value or, otherwise, by itself. An example is git add --verbose file1.
Property Type Description Required
type string (enum) Type of the argument. Choose between [subcommand, positional, optional] ✔️
description string Short description of what the tool does ✔️
id string Identifier of the argument (used only when template property is defined) ✅
value any type The value of the argument
option string The option name (only applicable for optional arguments). See Special option values section for additional customization ✅ (for optional arguments)
format string (enum) (for API clients) the value's data type. Useful for graphical interfaces, provides the means to render the form elements (inputs, text areas, dropdowns, ..). Choose between [string, array] ✅ (for API clients)
selectable boolean (for API clients) Whether the argument's value shall be customized by the user, or otherwise it is a fixed (non-modifiable) value. Non-selectable arguments are always set ✅ (for API clients)
repeatable boolean (for API clients) Whether the same argument can be used several times ✅ (for API clients)
explicit_paths boolean (for API clients) Only applicable for the assessment, whether the value property shall be set with the explicit list of files detected in the repository ✅ (for API clients)
args object Suitable when defining commands with more than one type of argument, it allows to define nested args properties

Special option values

The name of the option property might tell the SQAaaS API to perform additional work. So far it is only supported to set credentials when the option value contains the following substrings:

  • jenkins-credential-id, such as in --kubeconfig-jenkins-credential-id.
  • jenkins-credential-variable, such as in --kubeconfig-jenkins-credential-variable.

Reporting (reporting property)

The reporting property provides the pointers to the suitable output validator that is capable of parsing the data produced by the given tool. The list of available validator plugins officially supported in the SQAaaS platform can be found in the sqaaas-reporting-plugins repository.

The set of sub-properties that are meaningful to the reporting property are the following:

Property Type Description Required
validator string Id of the validator (the report2sqaaas CLI provides the possible choices) ✔️
threshold int The threshold that sets the output as valid or invalid ✅
requirement_level string (enum) Sets the relevance of the tool in regards to the fulfillment of the associated criterion. Choose between [REQUIRED, RECOMMENDED, OPTIONAL] ✅

Note:

  • The validator and threshold properties are passed as input arguments to the report2sqaaas module, and thus, they must be aligned/match those from such module.
  • The requirement_level of the tools is OPTIONAL by default (if no value is provided). Both [REQUIRED, RECOMMENDED] tools are processed by the QAA module, but only the REQUIRED are considered to determine the fulfillment of the criterion.

The default property

The tools property has a special key named default. Here you can define the tools that shall be available for all the defined criteria (in the criteria property).

Criteria properties

The SQA criteria1 that the SQAaaS platform is compliant with are identified by codes, such as QC.Sty for styling conventions or QC.Lic for the requirements related to the availability and maintenance of a source code license. The criteria key maps the tools (the tools key defined in the previous section) with the SQA criteria:

"criteria": {
    "QC.Sty": {
        "description": {
            "msg": "Use code style standards to guide your code writing so you let others understand it",
            "improves": "readability, reusability",
            "docs": "https://indigo-dc.github.io/sqa-baseline/#code-style-qc.sty"
        },
        "tools": {
            "Dockerfile": ["hadolint"],
            "JSON": ["jsonlint"],
            "Python": ["tox", "pycodestyle"]
        }
    },
    "QC.Lic": {
        "description": {
            "msg": "Usage of an open license to distribute your code",
            "improves": "external contributions, reusability",
            "docs": "https://indigo-dc.github.io/sqa-baseline/#licensing-qc.lic"
        },
        "tools": {
            "license": ["licensee"]
        }
    }
}

Each criterion has two properties: description and tools, which are defined as follows:

  • The tools object contains one element per language being supported. The values are a list of tools from the ones defined in the tools property from the previous section.
    • Note that the tools marked as default will be automatically added. There is no need to define those in the criterion data.
  • The description object is meant to be used by the SQAaaS API clients. It provides information about the criterion that is suitable to contextualize it within the user interface.

1Online versions: software and service's SQA criteria.