Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

simplify prepare_if_dev explanation in dbt tutorial #23765

Merged
merged 1 commit into from
Sep 12, 2024
Merged

simplify prepare_if_dev explanation in dbt tutorial #23765

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 3 additions & 8 deletions docs/content/integrations/dbt/using-dbt-with-dagster/load-dbt-models.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,16 +124,11 @@ dbt_project = DbtProject(
dbt_project.prepare_if_dev()
```

Invoking `dbt_project.prepare_if_dev()` prepares your dbt project for local development by creating the `manifest.json` at run time in `dagster dev`.
Generating the `manifest.json` file for a dbt project is time-consuming, so it's best to avoid doing so every time this Python module is imported. Thus, in production deployments of Dagster, you'll typically have the CI/CD system that packages up your code generate your `manifest.json`.

To disable this behavior during local development, remove `dbt_project.prepare_if_dev()`.
However, in development, you typically want changes made to files in your dbt project to be immediately reflected in the Dagster UI without needing to regenerate the manifest.

Based on whether `dbt_project.prepare_if_dev()` and `dagster dev` are used, either:

1. **This code generates the `manifest.json` for you.** This is the easiest option during development, because you never need to worry about the file being out-of-date with your dbt project. But generating the file can be time-consuming when you have a large dbt project, so it's generally recommended to go with the second option when you're deploying your dbt project to production.
2. **This code leaves it up to you to generate the `manifest.json` file on your own, and this code just reads it.** This option allows you to only generate it once every time you deploy, rather than every time the code in this module gets executed.

In this tutorial, we're going with the first option: We run `dagster dev` and use `dbt_project.prepare_if_dev()` to prepare the dbt project and create the manifest.
`dbt_project.prepare_if_dev()` helps with this – it re-generates your `manifest.json` at the time Dagster imports your code, _but_ only if it's being imported by the `dagster dev` command.

Once you've got a `manifest.json` file, it's time to define your Dagster assets using it. The following code, in your project's `assets.py`, does this:

Expand Down