Projects creation API does not support multiple pipeline options correctly

[AyanSinhaMahapatra]

There is a bug preventing running a map_deploy_to_develop pipeline with multiple options enabled.

When trying to create a project from the API like this:

import requests

api_url = "http://127.0.0.1:8001/api/projects/"
data = {
    "name": "d2d_test_v3",
    "input_urls": [
        "https://repo1.maven.org/maven2/javax/portlet/portlet-api/3.0.1/portlet-api-3.0.1.jar#to",
        "https://repo1.maven.org/maven2/javax/portlet/portlet-api/3.0.1/portlet-api-3.0.1-sources.jar#from"
    ],
    "pipeline": ["map_deploy_to_develop:Java,JavaScript"],
    "execute_now": True,
}

response = requests.post(api_url, data=data)
response.json()

We get the following error:

{'pipeline': ['"JavaScript" is not a valid choice.']}

But if we use one option like the following:

• "pipeline": ["map_deploy_to_develop:Java"]
• "pipeline": ["map_deploy_to_develop:JavaScript"]

It runs the pipeline with the specified optional steps selected without any issues.
So the pipeline option specified do exist and specifying options work nicely through the API for a single pipeline option, but fails for multiple pipeline options.

This is across any pipelines, i.e. "pipeline": ["resolve_dependencies:StaticResolver,DynamicResolver"], fails too with {'pipeline': ['"DynamicResolver" is not a valid choice.']}.

The same issue is not present in the CLI. When I run docker compose exec -it web scanpipe create-project test-cli --pipeline resolve_dependencies:StaticResolver,DynamicResolver --input-url <INPUT_URL> --execute it creates and runs the project with the specified selected options allright.

Relevant docs: https://scancodeio.readthedocs.io/en/latest/rest-api.html#create-a-project

[tdruez]

There is a syntax conflict as the comma , is used for both separating the pipeline names and optional steps.

["map_deploy_to_develop:Java,JavaScript"] ends up splitted as those 2 pipelines:
['map_deploy_to_develop:Java', 'JavaScript'] where it should be a single pipeline with 2 optional steps.

We may have to drop the support for providing multiple pipelines as a comma-separated string, such as "analyze_docker_image,scan_single_package", to only support providing a list.
Alternatively, we could change the separator for multiple optional steps, such as "map_deploy_to_develop:Java+JavaScript"

The same issue is not present in the CLI.

Because in the CLI, each pipeline is provided as an option and that string is easily parsed: create-project test-cli1 --pipeline resolve_dependencies:StaticResolver,DynamicResolver --pipeline map_deploy_to_develop:Java,JavaScript

---

I've introduced that bug recently in d5273cb#diff-e1ee2ed7559ce1892ae462df0375df930a4c51e037809e80a550df3374911bdcR87-R88
The simplest solution is to drop the "support for providing pipeline names as a comma-separated single string" as it was introduced very recently.
Changing the syntax for providing optional steps will definitively have more impact on users.

[AyanSinhaMahapatra]

The simplest solution is to drop the "support for providing pipeline names as a comma-separated single string" as it was introduced very recently.
Changing the syntax for providing optional steps will definitively have more impact on users.

This makes sense @tdruez , thanks!

[AyanSinhaMahapatra]

Closing as this is fixed!