Skip to content

Commit

Permalink
Change parent snapshot ID invariant to a warning
Browse files Browse the repository at this point in the history
Summary:
After https://github.com/dagster-io/dagster/pull/20335/files, there are edge cases where the parent snapshot ID returned in code might not match up with the parent job snapshot ID stored in the database or in memory for the full job. Rather than invarianting and failing the launched job, warn.

Test Plan:
- Run a grpc server locally, start dagit
- Add a new asset to the job, restart the gRPC server (may need to disable the watcher on dagit that automatically reloads the code location)
- Launch a subsetted job

Before: invariant stops job from launching
Now: warning, job launches just fine
  • Loading branch information
gibsondan committed Mar 14, 2024
1 parent 0d7ea9e commit e5b3f08
Showing 1 changed file with 9 additions and 7 deletions.
16 changes: 9 additions & 7 deletions python_modules/dagster/dagster/_core/instance/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -1277,14 +1277,16 @@ def _ensure_persisted_job_snapshot(
check.opt_inst_param(parent_job_snapshot, "parent_job_snapshot", JobSnapshot)

if job_snapshot.lineage_snapshot:
if not self._run_storage.has_job_snapshot(
job_snapshot.lineage_snapshot.parent_snapshot_id
):
returned_job_snapshot_id = self._run_storage.add_job_snapshot(
parent_job_snapshot # type: ignore # (possible none)
parent_snapshot_id = create_job_snapshot_id(check.not_none(parent_job_snapshot))

if job_snapshot.lineage_snapshot.parent_snapshot_id != parent_snapshot_id:
warnings.warn(
f"Stored parent snapshot ID {parent_snapshot_id} did not match the parent snapshot ID {job_snapshot.lineage_snapshot.parent_snapshot_id} on the subsetted job"
)
check.invariant(
job_snapshot.lineage_snapshot.parent_snapshot_id == returned_job_snapshot_id

if not self._run_storage.has_job_snapshot(parent_snapshot_id):
self._run_storage.add_job_snapshot(
check.not_none(parent_job_snapshot), parent_snapshot_id
)

job_snapshot_id = create_job_snapshot_id(job_snapshot)
Expand Down

0 comments on commit e5b3f08

Please sign in to comment.