Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Type checking issue on Dagster 1.5 #49

Open
Slasher154 opened this issue Oct 24, 2023 · 0 comments
Open

Type checking issue on Dagster 1.5 #49

Slasher154 opened this issue Oct 24, 2023 · 0 comments

Comments

@Slasher154
Copy link

Slasher154 commented Oct 24, 2023

Hi,

We recently updated Dagster to version 1.5 and found an following error while importing Meltano resource

2023-10-24 18:26:38 +0700 - dagster.code_server - ERROR - Error while importing code
Traceback (most recent call last):
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster/_grpc/server.py", line 295, in __init__
    self._loaded_repositories: Optional[LoadedRepositories] = LoadedRepositories(
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster/_grpc/server.py", line 139, in __init__
    loadable_targets = get_loadable_targets(
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster/_grpc/utils.py", line 50, in get_loadable_targets
    else loadable_targets_from_python_module(module_name, working_directory)
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster/_core/workspace/autodiscovery.py", line 35, in loadable_targets_from_python_module
    module = load_python_module(
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster/_core/code_pointer.py", line 135, in load_python_module
    return importlib.import_module(module_name)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/projects/my-projects/dagster-project/dagster_project/__init__.py", line 3, in <module>
    from .assets import dbt, dbt_assets, meltano_assets
  File "/home/projects/my-projects/dagster-project/dagster_project/assets.py", line 10, in <module>
    from dagster_meltano import meltano_resource
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster_meltano/__init__.py", line 3, in <module>
    from dagster_meltano.generation import (
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster_meltano/generation.py", line 8, in <module>
    from dagster_meltano.meltano_resource import MeltanoResource
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster_meltano/meltano_resource.py", line 12, in <module>
    from dagster_meltano.job import Job
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster_meltano/job.py", line 18, in <module>
    from dagster_meltano.ops import meltano_run_op as meltano_run_op_factory
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster_meltano/ops.py", line 131, in <module>
    def meltano_install_op(context: OpExecutionContext):
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster/_core/definitions/decorators/op_decorator.py", line 125, in __call__
    op_def = OpDefinition.dagster_internal_init(
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster/_core/definitions/op_definition.py", line 205, in dagster_internal_init
    return OpDefinition(
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster/_core/decorator_utils.py", line 195, in wrapped_with_pre_call_fn
    return fn(*args, **kwargs)
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster/_core/definitions/op_definition.py", line 151, in __init__
    _validate_context_type_hint(self._compute_fn.decorated_fn)
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster/_core/definitions/op_definition.py", line 576, in _validate_context_type_hint
    raise DagsterInvalidDefinitionError(
dagster._core.errors.DagsterInvalidDefinitionError: Cannot annotate `context` parameter with type OpExecutionContext. `context` must be annotated with AssetExecutionContext, OpExecutionContext, or left blank.
/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster/_core/workspace/context.py:616: UserWarning: Error loading repository location dagster_project:dagster._core.errors.DagsterInvalidDefinitionError: Cannot annotate `context` parameter with type OpExecutionContext. `context` must be annotated with AssetExecutionContext, OpExecutionContext, or left blank.
Stack Trace:
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster/_grpc/server.py", line 295, in __init__
    self._loaded_repositories: Optional[LoadedRepositories] = LoadedRepositories(
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster/_grpc/server.py", line 139, in __init__
    loadable_targets = get_loadable_targets(
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster/_grpc/utils.py", line 50, in get_loadable_targets
    else loadable_targets_from_python_module(module_name, working_directory)
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster/_core/workspace/autodiscovery.py", line 35, in loadable_targets_from_python_module
    module = load_python_module(
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster/_core/code_pointer.py", line 135, in load_python_module
    return importlib.import_module(module_name)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/projects/my-projects/dagster-project/dagster_project/__init__.py", line 3, in <module>
    from .assets import dbt, dbt_assets, meltano_assets
  File "/home/projects/my-projects/dagster-project/dagster_project/assets.py", line 10, in <module>
    from dagster_meltano import meltano_resource
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster_meltano/__init__.py", line 3, in <module>
    from dagster_meltano.generation import (
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster_meltano/generation.py", line 8, in <module>
    from dagster_meltano.meltano_resource import MeltanoResource
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster_meltano/meltano_resource.py", line 12, in <module>
    from dagster_meltano.job import Job
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster_meltano/job.py", line 18, in <module>
    from dagster_meltano.ops import meltano_run_op as meltano_run_op_factory
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster_meltano/ops.py", line 131, in <module>
    def meltano_install_op(context: OpExecutionContext):
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster/_core/definitions/decorators/op_decorator.py", line 125, in __call__
    op_def = OpDefinition.dagster_internal_init(
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster/_core/definitions/op_definition.py", line 205, in dagster_internal_init
    return OpDefinition(
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster/_core/decorator_utils.py", line 195, in wrapped_with_pre_call_fn
    return fn(*args, **kwargs)
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster/_core/definitions/op_definition.py", line 151, in __init__
    _validate_context_type_hint(self._compute_fn.decorated_fn)
  File "/home/projects/my-projects/.venv/lib/python3.10/site-packages/dagster/_core/definitions/op_definition.py", line 576, in _validate_context_type_hint
    raise DagsterInvalidDefinitionError(
  warnings.warn(f"Error loading repository location {location_name}:{error.to_string()}")

Based on a Dagster Slack discussion it seems that Dagster version 1.5 added a type checking which results into this error. The Op manages to solve this error by removing following line from the code.

from __future__ import annotations
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant