Skip to content

Commit

Permalink
docs: consolidate setup config to pyproject.toml
Browse files Browse the repository at this point in the history
  • Loading branch information
dbrtly committed Oct 19, 2024
1 parent 88e1a24 commit d2ec720
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 22 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ We recommend defining ops and graphs a job file along with the job definition wi

## For multiple code locations

So far, we've discussed our recommendations for structuring a large project which contains only one code location. Dagster also allows you to structure a project with multiple definitions. We don't recommend over-abstracting too early; in most cases, one code location should be sufficient. A helpful pattern uses multiple code locations to separate conflicting dependencies, where each definition has its own package requirements (e.g., `setup.py`) and deployment specs (e.g., Dockerfile).
So far, we've discussed our recommendations for structuring a large project which contains only one code location. Dagster also allows you to structure a project with multiple definitions. We don't recommend over-abstracting too early; in most cases, one code location should be sufficient. A helpful pattern uses multiple code locations to separate conflicting dependencies, where each definition has its own package requirements (e.g., `pyproject.toml`) and deployment specs (e.g., Dockerfile).

To include multiple code locations in a single project, you'll need to add a [configuration file](/guides/understanding-dagster-project-files#configuration-files) to your project:

Expand Down
33 changes: 12 additions & 21 deletions docs/content/guides/understanding-dagster-project-files.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -131,8 +131,8 @@ Let's take a look at what each of these files and directories does:
<tr>
<td>pyproject.toml</td>
<td>
A file that specifies package core metadata in a static, tool-agnostic
way.
A file that specifies package configuration in a static, tool-agnostic
format.
<br />
<br />
This file includes a <code>tool.dagster</code> section which references the
Expand All @@ -148,16 +148,10 @@ Let's take a look at what each of these files and directories does:
<a href="https://peps.python.org/pep-0518/">
PEP-518
</a>{" "}
and meant to replace <code>setup.py</code>, but we may still include a{" "}
<code>setup.py</code> for compatibility with tools that do not use this
and meant to replace <code>setup.py</code>, but we may still include {" "}
<code>setup.cfg</code> and {" "}
<code>setup.py</code> files for compatibility with tools that do not use this
spec.
</td>
</tr>
<tr>
<td>setup.py</td>
<td>
A build script with Python package dependencies for your new project as
a package. Use this file to specify dependencies.
<br /><br />
<strong>Note</strong>: If using Dagster+, add <code>dagster-cloud</code> as a dependency.
</td>
Expand All @@ -169,6 +163,13 @@ Let's take a look at what each of these files and directories does:
commands.
</td>
</tr>
<tr>
<td>setup.py</td>
<td>
A optional build script for your new project that specifies Python package dependencies
and project setup. Some developers and tooling may work best with this file.
</td>
</tr>
</tbody>
</table>

Expand Down Expand Up @@ -301,8 +302,6 @@ For local development, a project with a single code location might look like thi
├── my_dagster_project_tests
├── dagster.yaml ## optional, used for instance settings
├── pyproject.toml ## optional, used to define the project as a module
├── setup.cfg
├── setup.py
└── tox.ini
```

Expand All @@ -323,8 +322,6 @@ For local development, a project with multiple code locations might look like th
├── my_dagster_project_tests
├── dagster.yaml ## optional, used for instance settings
├── pyproject.toml
├── setup.cfg
├── setup.py
├── tox.ini
└── workspace.yaml ## defines multiple code locations
```
Expand All @@ -350,8 +347,6 @@ A Dagster project deployed to your infrastructure might look like this:
├── my_dagster_project_tests
├── dagster.yaml ## optional, used for instance settings
├── pyproject.toml
├── setup.cfg
├── setup.py
├── tox.ini
└── workspace.yaml ## defines multiple code locations
```
Expand All @@ -378,8 +373,6 @@ For a Dagster+ Serverless deployment, a project might look like this:
├── dagster_cloud.yaml ## defines code locations
├── deployment_settings.yaml ## optional, defines settings for full deployments
├── pyproject.toml
├── setup.cfg
├── setup.py
└── tox.ini
```

Expand All @@ -402,8 +395,6 @@ For a Dagster+ Hybrid deployment, a project might look like this:
├── dagster_cloud.yaml ## defines code locations
├── deployment_settings.yaml ## optional, defines settings for full deployments
├── pyproject.toml
├── setup.cfg
├── setup.py
└── tox.ini
```

Expand Down

0 comments on commit d2ec720

Please sign in to comment.