Skip to content

Commit

Permalink
[components] Add execute method to dbt project, remove resource from …
Browse files Browse the repository at this point in the history
…defs (#26363)

## Summary & Motivation

As title -- we already have a reference to the dbt resource from the
component loading codepath, and so threading it through Definitions
doesn't actually make much sense.

It's free to construct one of these things (it doesn't have to do any
expensive spinup), and this way we don't need to worry about situations
where we have multiple dbt project components with conflicting resource
keys

Other integrations where resource construction is not free will not be
able to use this pattern, but this sort of thing is quite common

## How I Tested These Changes

## Changelog

NOCHANGELOG
  • Loading branch information
OwenKephart authored Dec 11, 2024
1 parent d8fda21 commit 0fe2d5e
Showing 1 changed file with 7 additions and 4 deletions.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import os
from pathlib import Path
from typing import Any, Mapping, Optional
from typing import Any, Iterator, Mapping, Optional

import click
import dagster._check as check
Expand Down Expand Up @@ -109,10 +109,10 @@ def build_defs(self, context: ComponentLoadContext) -> Definitions:
op_tags=self.op_spec.tags if self.op_spec else None,
dagster_dbt_translator=self.dbt_translator,
)
def _fn(context: AssetExecutionContext, dbt: DbtCliResource):
yield from dbt.cli(["build"], context=context).stream()
def _fn(context: AssetExecutionContext):
yield from self.execute(context=context, dbt=self.dbt_resource)

return Definitions(assets=[_fn], resources={"dbt": self.dbt_resource})
return Definitions(assets=[_fn])

@classmethod
def generate_files(cls, params: DbtGenerateParams) -> Mapping[str, Any]:
Expand All @@ -135,3 +135,6 @@ def generate_files(cls, params: DbtGenerateParams) -> Mapping[str, Any]:
relative_path = None

return {"dbt": {"project_dir": relative_path}}

def execute(self, context: AssetExecutionContext, dbt: DbtCliResource) -> Iterator:
yield from dbt.cli(["build"], context=context).stream()

0 comments on commit 0fe2d5e

Please sign in to comment.