Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KeyError in exposure in dbt cloud #26648

Open
bramrodenburg opened this issue Dec 20, 2024 · 0 comments
Open

KeyError in exposure in dbt cloud #26648

bramrodenburg opened this issue Dec 20, 2024 · 0 comments
Labels
integration: dbt Related to dagster-dbt type: bug Something isn't working

Comments

@bramrodenburg
Copy link

What's the issue?

A Dagster job that references a dbt Cloud job keeps failing on multiple dbt exposures. Each time we remove the exposure causing the failure, the job fails again on a different exposure.

Error message:

dagster._core.errors.DagsterExecutionStepExecutionError: Error occurred while executing op "dbt_cloud_job_453028":

  File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/plan/execute_plan.py", line 245, in dagster_event_sequence_for_step
    for step_event in check.generator(step_events):
  File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/plan/execute_step.py", line 499, in core_dagster_event_sequence_for_step
    for user_event in _step_output_error_checked_user_event_sequence(
  File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/plan/execute_step.py", line 183, in _step_output_error_checked_user_event_sequence
    for user_event in user_event_sequence:
  File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/plan/execute_step.py", line 87, in _process_asset_results_to_events
    for user_event in user_event_sequence:
  File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/plan/compute.py", line 193, in execute_core_compute
    for step_output in _yield_compute_results(step_context, inputs, compute_fn, compute_context):
  File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/plan/compute.py", line 162, in _yield_compute_results
    for event in iterate_with_context(
  File "/usr/local/lib/python3.10/site-packages/dagster/_utils/__init__.py", line 480, in iterate_with_context
    with context_fn():
  File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/plan/utils.py", line 84, in op_execution_error_boundary
    raise error_cls(

The above exception was caused by the following exception:
KeyError: 'exposure.gigs_analytics.ld_chart_0f51de6a_60b0_4013_86bd_dc384d2ab5ee'

  File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/plan/utils.py", line 54, in op_execution_error_boundary
    yield
  File "/usr/local/lib/python3.10/site-packages/dagster/_utils/__init__.py", line 482, in iterate_with_context
    next_output = next(iterator)
  File "/usr/local/lib/python3.10/site-packages/dagster_dbt/cloud/asset_defs.py", line 560, in _assets
    yield from result_to_events(
  File "/usr/local/lib/python3.10/site-packages/dagster_dbt/cloud/utils.py", line 107, in result_to_events
    manifest_json["nodes"][unique_id] if manifest_json else {"unique_id": unique_id}

What did you expect to happen?

The Dagster job to succeed (in dbt cloud it does succeed)

How to reproduce?

No response

Dagster version

1.9.2

Deployment type

Dagster Helm chart

Deployment details

Dagster running on Google Kubernetes Engine (GKE)

Additional information

Version of Dagster/dbt: `dagster-dbt==0.25.2

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.
By submitting this issue, you agree to follow Dagster's Code of Conduct.

@bramrodenburg bramrodenburg added the type: bug Something isn't working label Dec 20, 2024
@garethbrickman garethbrickman added the integration: dbt Related to dagster-dbt label Dec 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
integration: dbt Related to dagster-dbt type: bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants