How to change one component for another at build time?

[sreinken-va]

Hello! I am trying to create two different versions of a cli tool for two clients, but both should use the same base, with different components. I know how to do this in the Clojure implementation of Polylith, but trying to do so with python has eluded me.

I tried to structure my project like so:

bases/
  mynamespace/
    cli/
      __init__.py
      core.py
components/
  mynamespace/
    cli_impl/
      __init__.py
      core.py
  client1/
    mynamespace/
      cli_impl/
        __init__.py
       core.py
  client2/
    mynamespace/
      cli_impl/
        __init__.py
       core.py
projects/
  client1_project/
    pyproject.toml
  client2_project/
    pyproject.toml

The component under the high level namespace would have shared code, while client specific code would be under the folders for each client.

In the base, I would do an import like so:

from mynamespace import cli_impl

cli_impl.do_something()

And then in the pyproject.toml for each client, I would try the following.

client1's pyproject.toml

[tool.poetry]
name = "client1-cli"

packages = [
    {include = "mynamespace/cli", from = "../../bases"},
    {include = "mynamespace/cli_impl", from = "../../components/client1"},
]

client2's pyproject.toml

[tool.poetry]
name = "client2-cli"

packages = [
    {include = "mynamespace/cli", from = "../../bases"},
    {include = "mynamespace/cli_impl", from = "../../components/client2"},
]

I expected this to be the best way to change out one component for another at build time, but it didn't work, and it continues to go outside of the client1 and client2 subfolders when building. Is it possible to do this? Am I going about it the wrong way?

[DavidVujic]

I think this failes because of the way bricks are identified currently. It is pretty strict; the tool expects the flat namespace/brick structure.

I wasn't aware about this possibility in Clojure - I know about profiles and swappable components, but have thought of that as advanced setup and for local testing and development, such as swapping out a component (that in production is doing inserts in a DB) for an in-memory one or something similar. I will read up on this more. I wasn't aware of the possibility to sub-namespace them like this.

What I would do is to instead have two separate thin bases for each client, and let them import their respective components. Would that work for you? Also, wouldn't the client1 and client2 be identified as the bricks as it is set up now (and not the sub-namespaces)?

Another alternative would be to have one component, but two "endpoints" in it, one for each client, unless they require different third-party libraries.

[sreinken-va]

I suppose this means that you can't reuse bases in different projects, since that would require this feature, otherwise you would need a place in code that references both projects.. Also, in my case there is a good chance the two clients may need different third party libraries.

I was trying to follow the namespace packaging idea from here: https://packaging.python.org/en/latest/guides/packaging-namespace-packages/

I was also doing research here: https://peps.python.org/pep-0420/

If we could find a way to do this, it would be EXTREMELY helpful to me, as it would greatly strengthen my business case for using Polylith. I think this means of Python namespace packaging seems the most logical, as it might work with the include paths in the project tomls

[DavidVujic]

I think it should work to do as you have done from a tech perspective, but I might have made assumptions about the flat structure as I mentioned before. I can have a look at it!

From a Polylith perspective, I would still recommend to rethink the setup with the nested client1 and client2. Are they represented as components when you run poly info? Ideally, I think the cli_impl should be the components and not the top folders.

As I see it, the reusability of bases are not as common as for components - even if it is possible. The main use case that I have known about before for reusing a base is to have several services running with the same base but with different environment variables.

If there will be code duplications in the bases when creating two different for the clients, there might be an opportunity to refactor the duplicated code into reusable components. I hope this helps!

[sreinken-va]

It's interesting. The point of reusing the bases is that I want the CLI commands and interface to be the exact same (this cannot really be moved to a component in a meaningful way), but it calls different backing services.

If I do this with Environment Variables, then I would still have to import from ALL of the components with ALL possible implementations, even if they never get used, locked behind environment vars. When I ship to both clients, the tool would end up much larger than it needs to be, and including dependencies that I would prefer not be there. If that's the only method by which I can accomplish this right now, I suppose I can go forward with it.

Also, from the Polylith Perspective, I would also prefer client1 and client2 to be ordinary components. I only nested them like that because it was the only way I found for Python to locate a module across more than 1 file. poly info does not recognize them as components.

Thanks for the help though! I appreciate you taking the time to look at this.