Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add legacy namespace packages to airflow.providers #47064

Merged

Conversation

potiuk
Copy link
Member

@potiuk potiuk commented Feb 25, 2025

It seem that vscode cannot recognise cross-provider imports when providers have implicit namespaces for "airflow.providers" and "airflow.providers.nnnn" (for "common" etc.). And what seem to halp is to add legacy namespace __init__.py files for those packages.

This has no impact on generated packages - because generated packages have "tool.flit.module" set to the right sub-package and flit will not use the init.py files when packages are generated.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@potiuk
Copy link
Member Author

potiuk commented Feb 25, 2025

Generated contents of the .whl distributions:

Archive:  dist/apache_airflow_providers_apache_beam-6.0.2-py3-none-any.whl
    testing: airflow/providers/apache/beam/LICENSE   OK
    testing: airflow/providers/apache/beam/README.md   OK
    testing: airflow/providers/apache/beam/__init__.py   OK
    testing: airflow/providers/apache/beam/get_provider_info.py   OK
    testing: airflow/providers/apache/beam/hooks/__init__.py   OK
    testing: airflow/providers/apache/beam/hooks/beam.py   OK
    testing: airflow/providers/apache/beam/operators/__init__.py   OK
    testing: airflow/providers/apache/beam/operators/beam.py   OK
    testing: airflow/providers/apache/beam/triggers/__init__.py   OK
    testing: airflow/providers/apache/beam/triggers/beam.py   OK
    testing: apache_airflow_providers_apache_beam-6.0.2.dist-info/entry_points.txt   OK
    testing: apache_airflow_providers_apache_beam-6.0.2.dist-info/WHEEL   OK
    testing: apache_airflow_providers_apache_beam-6.0.2.dist-info/METADATA   OK
    testing: apache_airflow_providers_apache_beam-6.0.2.dist-info/RECORD   OK

Both airflow.providers and apache.providers.apache are still implicit packages after building the packages - even if they are legacy packages in the sources.

@potiuk potiuk force-pushed the add-legacy-namespace-package-to-providers branch 3 times, most recently from 723fb82 to bdcae41 Compare February 25, 2025 15:50
@potiuk potiuk force-pushed the add-legacy-namespace-package-to-providers branch 2 times, most recently from b901d04 to cf1beb4 Compare February 25, 2025 19:40
Copy link
Contributor

@jscheffl jscheffl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aaaah, cool. Finally after re-structuring the syntax highlighting in VSCode works again!

@potiuk
Copy link
Member Author

potiuk commented Feb 25, 2025

Aaaah, cool. Finally after re-structuring the syntax highlighting in VSCode works again!

Indeed looks like :)

It seem that vscode cannot recognise cross-provider imports when
providers have implicit namespaces for "airflow.providers" and
"airflow.providers.nnnn" (for "common" etc.). And what seem
to halp is to add legacy namespace `__init__.py` files for those
packages.

This has no impact on generated packages - because generated
packages have "tool.flit.module" set to the right sub-package
and flit will not use the __init__.py files when packages are
generated.
@potiuk potiuk force-pushed the add-legacy-namespace-package-to-providers branch from cf1beb4 to a847bc5 Compare February 25, 2025 20:21
@potiuk
Copy link
Member Author

potiuk commented Feb 25, 2025

Random failure. Merging :)

@potiuk potiuk merged commit c6c4f95 into apache:main Feb 25, 2025
88 of 89 checks passed
@potiuk potiuk deleted the add-legacy-namespace-package-to-providers branch February 25, 2025 21:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants