Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use __dependents_rules__ to optimize pants dependents calculation #21619

Open
grihabor opened this issue Nov 6, 2024 · 0 comments
Open

Use __dependents_rules__ to optimize pants dependents calculation #21619

grihabor opened this issue Nov 6, 2024 · 0 comments

Comments

@grihabor
Copy link
Contributor

grihabor commented Nov 6, 2024

Is your feature request related to a problem? Please describe.
We have the following monorepo structure:

src/python/ # shared code
project1/
project2/
...

Projects are not allowed to depend on different project targets except for src/python, so we have rules for every project:

__dependents_rules__(
    # We don't want anybody to import from project1.
    (
        {"path": "project1/**"},
        "project1/**",
        "!*",
    ),
    ("*", "!*"),
)

Every project has it's own docker_image targets and helm_deployment targets.
Now, imagine we change some code in project1/src/ and try to deploy it with:

pants dependents --transitive --changed-since=origin/main

This triggers a bunch of calculation like:

⠁ 0.00s Rendering Helm deployment project1/helm:project1
⠁ 1.03s Rendering Helm deployment project2/helm:project2
⠁ 1.45s Rendering Helm deployment project3/helm:project3
⠁ 1.52s Rendering Helm deployment project4/helm:project4
⠁ 0.54s Rendering Helm deployment project5/helm:project5
⠁ 1.61s Rendering Helm deployment project6/helm:project6
⠁ 0.09s Rendering Helm deployment project7/helm:project7

which takes about 4 minutes without cache.

However we know for sure that no project can depend on project1/src/ which means we can skip other projects.

Describe the solution you'd like
Collect information from __dependents_rules__ and use it to optimize pants dependents logic.

Describe alternatives you've considered
Live with very slow pants dependents and cry

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant