Migrating Ecoindex from micro services to polylith...

[vvatelot]

Hello, I am trying to migrate ecoindex project (composed from ecoindex_python, ecoindex_scraper, ecoindex_cli and ecoindex_api) from micro services to a polylith project using python-polylith. I understand the theory, but in practice, I think I must miss something 😄

I started this project yesterday https://github.com/vvatelot/ecoindex_monorepo and for now I am just trying to implement the basic components (ie compute) and run related tests.

What I understand I have to do:

• Install deps at the top level using poetry install
• Run tests using poetry run pytest test/
• Or run an example script with poetry run python development/ecoindex_compute.py

But here are the results:

Example development script:

Traceback (most recent call last):
  File "/home/20100229/Devel/perso/ecoindex/ecoindex_monorepo/development/ecoindex_compute.py", line 1, in <module>
    from ecoindex.models import Ecoindex
ModuleNotFoundError: No module named 'ecoindex'

Pytest:

_____________________________________________________ ERROR collecting test/components/ecoindex/compute/test_models.py _____________________________________________________
ImportError while importing test module '/home/20100229/Devel/perso/ecoindex/ecoindex_monorepo/test/components/ecoindex/compute/test_models.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
test/components/ecoindex/compute/test_models.py:4: in <module>
    from ecoindex.models import Ecoindex, Result, ScreenShot, WebPage
E   ModuleNotFoundError: No module named 'ecoindex'
[...]

What am I doing wrong ? 🤔

[DavidVujic]

Hi @vvatelot!

Glad that you are giving Polylith a try! I had a quick look at your repo and don't see anything wrong. I will clone your repo to look what might be wrong.

[DavidVujic]

Hello again @vvatelot!

I think I have found the issue, and it has something to do with pkg_resources. I tried out running the code in the development folder in a REPL and found the cause for the error:

It was the call to get_distribution https://github.com/vvatelot/ecoindex_monorepo/blob/main/components/ecoindex/models/compute.py#L39

I haven't any experience with pkg_resources and what it does, but it seems that importlib is a replacement?

Here's what I tried out:

Here I'm using a local virtual environment (recommended) with the poetry.toml confi.

Here, being able to run the code in the development folder:

[DavidVujic]

I'm just guessing: "get distribution", is that about extracting information from an imported package? If so maybe that is why the call fail, because it looks like it is doing a lookup on a (third party) package that doesn't exist.

[vvatelot]

Hello @DavidVujic thank you so much for your quick response!
It works now! And yes, I was using get_distribution to get the version of an external package... It worked... until I move to polylith which is completely normal as I am now using ecoindex_compute as a components which has no version. I will find another way to do it.

But anyway, it works, I will continue this work!

[DavidVujic]

Great!

For a great developer experience, I recommend to have a local virtual environment. Also, If you use tools like MyPy or PyRight, I recommend to add configuration so the code editor understand the Polylith structure better. I have written about it here: https://davidvujic.github.io/python-polylith-docs/ide/

[vvatelot]

Yes! I had already this configuration applied globally, but you are right, this is better to explicit it here.