Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up build #64

Merged
merged 7 commits into from
Mar 19, 2024
Merged

Clean up build #64

merged 7 commits into from
Mar 19, 2024

Conversation

cnolanminich
Copy link
Contributor

this PR does a bunch of small things to the build process for the hooli project:

  • adds uv as the package installer (should ~halve the build time)
  • remove unused or duplicate python package installations
  • bumps the python version to 3.11 in the Dockerfile
  • sets an upper bound in setup.py to <3.12, in part because there are issues loading Dagster on 3.12 that I didn't investigate

@cnolanminich cnolanminich requested review from slopp and izzye84 March 8, 2024 19:02
@cnolanminich cnolanminich self-assigned this Mar 8, 2024
Copy link

github-actions bot commented Mar 8, 2024

Your pull request is automatically being deployed to Dagster Cloud.

Location Status Link Updated
demo_assets View in Cloud Mar 08, 2024 at 09:23 PM (UTC)
snowflake_insights View in Cloud Mar 08, 2024 at 09:23 PM (UTC)
basics View in Cloud Mar 08, 2024 at 09:23 PM (UTC)
data-eng-pipeline View in Cloud Mar 08, 2024 at 09:23 PM (UTC)
batch_enrichment View in Cloud Mar 08, 2024 at 09:23 PM (UTC)

@@ -6,9 +6,6 @@ clean:
rm -rf ~/.dagster_home; mkdir ~/.dagster_home; cp dagster.yaml ~/.dagster_home/dagster.yaml

manifest:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This command is run in the context of the GHA runner in order to build the manifest, so these installs are necessary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I realized that too late -- that said, I'd prefer to put the commands in the GHA vs. in the make file (since you might run make manifest in the context of local dev, in which case we don't need the pip installs (handled via the setup.py)

Copy link
Collaborator

@izzye84 izzye84 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - Sorry for the long delay (I will be better about these going forwrad!)

@cnolanminich cnolanminich merged commit f5ae6f3 into master Mar 19, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants