From d2ec720ebf20df80f473e330d34693a4ec458788 Mon Sep 17 00:00:00 2001 From: Daniel Bartley Date: Sat, 19 Oct 2024 00:32:48 +1100 Subject: [PATCH] docs: consolidate setup config to pyproject.toml --- .../dagster/recommended-project-structure.mdx | 2 +- .../understanding-dagster-project-files.mdx | 33 +++++++------------ 2 files changed, 13 insertions(+), 22 deletions(-) diff --git a/docs/content/guides/dagster/recommended-project-structure.mdx b/docs/content/guides/dagster/recommended-project-structure.mdx index e703426ae7c5a..ee7c23be291da 100644 --- a/docs/content/guides/dagster/recommended-project-structure.mdx +++ b/docs/content/guides/dagster/recommended-project-structure.mdx @@ -162,7 +162,7 @@ We recommend defining ops and graphs a job file along with the job definition wi ## For multiple code locations -So far, we've discussed our recommendations for structuring a large project which contains only one code location. Dagster also allows you to structure a project with multiple definitions. We don't recommend over-abstracting too early; in most cases, one code location should be sufficient. A helpful pattern uses multiple code locations to separate conflicting dependencies, where each definition has its own package requirements (e.g., `setup.py`) and deployment specs (e.g., Dockerfile). +So far, we've discussed our recommendations for structuring a large project which contains only one code location. Dagster also allows you to structure a project with multiple definitions. We don't recommend over-abstracting too early; in most cases, one code location should be sufficient. A helpful pattern uses multiple code locations to separate conflicting dependencies, where each definition has its own package requirements (e.g., `pyproject.toml`) and deployment specs (e.g., Dockerfile). To include multiple code locations in a single project, you'll need to add a [configuration file](/guides/understanding-dagster-project-files#configuration-files) to your project: diff --git a/docs/content/guides/understanding-dagster-project-files.mdx b/docs/content/guides/understanding-dagster-project-files.mdx index 22e0718cb6f49..23b9ef43b9480 100644 --- a/docs/content/guides/understanding-dagster-project-files.mdx +++ b/docs/content/guides/understanding-dagster-project-files.mdx @@ -131,8 +131,8 @@ Let's take a look at what each of these files and directories does: pyproject.toml - A file that specifies package core metadata in a static, tool-agnostic - way. + A file that specifies package configuration in a static, tool-agnostic + format.

This file includes a tool.dagster section which references the @@ -148,16 +148,10 @@ Let's take a look at what each of these files and directories does: PEP-518 {" "} - and meant to replace setup.py, but we may still include a{" "} - setup.py for compatibility with tools that do not use this + and meant to replace setup.py, but we may still include {" "} + setup.cfg and {" "} + setup.py files for compatibility with tools that do not use this spec. - - - - setup.py - - A build script with Python package dependencies for your new project as - a package. Use this file to specify dependencies.

Note: If using Dagster+, add dagster-cloud as a dependency. @@ -169,6 +163,13 @@ Let's take a look at what each of these files and directories does: commands. + + setup.py + + A optional build script for your new project that specifies Python package dependencies + and project setup. Some developers and tooling may work best with this file. + + @@ -301,8 +302,6 @@ For local development, a project with a single code location might look like thi ├── my_dagster_project_tests ├── dagster.yaml ## optional, used for instance settings ├── pyproject.toml ## optional, used to define the project as a module -├── setup.cfg -├── setup.py └── tox.ini ``` @@ -323,8 +322,6 @@ For local development, a project with multiple code locations might look like th ├── my_dagster_project_tests ├── dagster.yaml ## optional, used for instance settings ├── pyproject.toml -├── setup.cfg -├── setup.py ├── tox.ini └── workspace.yaml ## defines multiple code locations ``` @@ -350,8 +347,6 @@ A Dagster project deployed to your infrastructure might look like this: ├── my_dagster_project_tests ├── dagster.yaml ## optional, used for instance settings ├── pyproject.toml -├── setup.cfg -├── setup.py ├── tox.ini └── workspace.yaml ## defines multiple code locations ``` @@ -378,8 +373,6 @@ For a Dagster+ Serverless deployment, a project might look like this: ├── dagster_cloud.yaml ## defines code locations ├── deployment_settings.yaml ## optional, defines settings for full deployments ├── pyproject.toml -├── setup.cfg -├── setup.py └── tox.ini ``` @@ -402,8 +395,6 @@ For a Dagster+ Hybrid deployment, a project might look like this: ├── dagster_cloud.yaml ## defines code locations ├── deployment_settings.yaml ## optional, defines settings for full deployments ├── pyproject.toml -├── setup.cfg -├── setup.py └── tox.ini ```