Installing only some connectors from source fails

[sbidin]

Describe the bug
Following the instructions for installing only some connectors (not all defaults) from source results in an error.

$ make pipelinewise connectors -e pw_connector=tap-postgres,target-snowflake
... <snip> ...
ERROR: Directory not exists and does not look like a valid singer connector: singer-connectors: singer-connectors/tap-postgrestarget-snowflake

To Reproduce

1. run make pipelinewise connectors -e pw_connector=tap-postgres,target-snowflake
2. see error

Expected behavior
The separate connectors each get installed.

Your environment

• pipelinewise 0.45.0, commit 58b88e2

Additional context
Possibly the documentation is simply out of date? Attached screenshot. There is no --connectors argument (rather only the connectors subcommand). Specifying the connectors as separated by spaces (and quoted together) instead of by commas fixes the issue.

[Samira-El]

Hey, it's working fine by me, our end to end tests CI job is also installing only a subset of connectors here and they're working fine.

Seeing the bit singer-connectors/tap-postgrestarget-snowflake in the error you shared makes me assume either your OS or Make on your machine is not recognizing the comma separator so ends up concatenating the connector names as one.

What OS you're running this command on and what Make version do you have?

[sbidin]

Ah, then I suppose it's something about my setup. I'm using:

• Arch Linux, updated
• GNU Make 4.3
• Python 3.9.12 (including this because setup fails for me on 3.10+ due to other reasons)

[Samira-El]

Installation scripts are not tested on Arch linux.
I advise you to spin up Pipelinewise as a docker container, the official images are here

[gbatiz]

Had the same issue. (Also Arch linux, although I cant't see how it can be relevant for the problem).
My workaround was to use separate commands:

make pipelinewise connectors -e pw_connector=tap-mysql -e pw_acceptlicenses=y
make pipelinewise connectors -e pw_connector=target-snowflake -e pw_acceptlicenses=y
make pipelinewise connectors -e pw_connector=transform-field -e pw_acceptlicenses=y