Skip to content

Commit

Permalink
handle json parsing errors from dbt log (dagster-io#24194)
Browse files Browse the repository at this point in the history
## Summary & Motivation
Fixes bug where some raw dbt logs not getting handled correctly.

Blamerev: dagster-io#22559

## How I Tested These Changes
Inspection

## Changelog [Bug]
* [dagster-dbt] fix bug where some forms of raw dbt logs (empty
newlines) were not being handled correctly
prha authored Sep 4, 2024
1 parent 291ab22 commit 96caa6b
Showing 1 changed file with 5 additions and 4 deletions.
Original file line number Diff line number Diff line change
@@ -410,19 +410,20 @@ def _stream_stdout(self) -> Iterator[Union[str, Dict[str, Any]]]:

with self.process.stdout:
for raw_line in self.process.stdout or []:
raw_event = raw_line.decode().strip()
raw_event_str = raw_line.decode().strip()

try:
raw_event = orjson.loads(raw_event)
raw_event = orjson.loads(raw_event_str)

# Parse the error message from the event, if it exists.
is_error_message = raw_event["info"]["level"] == "error"
if is_error_message:
self._error_messages.append(raw_event["info"]["msg"])

yield raw_event
except:
pass
yield raw_event_str

yield raw_event
except DagsterExecutionInterruptedError:
logger.info(f"Forwarding interrupt signal to dbt command: `{self.dbt_command}`.")

0 comments on commit 96caa6b

Please sign in to comment.