diff --git a/docs/docs-beta/docs/dagster-plus/deployment/management/managing-compute-logs-and-error-messages.md b/docs/docs-beta/docs/dagster-plus/deployment/management/managing-compute-logs-and-error-messages.md index 01279273021f2..ec61dc4fbbee3 100644 --- a/docs/docs-beta/docs/dagster-plus/deployment/management/managing-compute-logs-and-error-messages.md +++ b/docs/docs-beta/docs/dagster-plus/deployment/management/managing-compute-logs-and-error-messages.md @@ -3,4 +3,106 @@ title: Managing compute logs and error messages unlisted: true --- -{/* TODO move from https://docs.dagster.io/dagster-plus/managing-deployments/controlling-logs */} \ No newline at end of file +import ThemedImage from '@theme/ThemedImage'; + +:::note +This guide is applicable to Dagster+. +::: + +In this guide, we'll cover how to adjust where Dagster+ compute logs are stored and manage masking of error messages in the Dagster+ UI. + +{/* By default, Dagster+ ingests [structured event logs and compute logs](/concepts/logging#log-types) from runs and surfaces error messages from [code locations](/guides/deploy/code-locations/) in the UI. */} +By default, Dagster+ ingests [structured event logs and compute logs](/todo) from runs and surfaces error messages from [code locations](/todo) in the UI. + +Depending on your organization's needs, you may want to retain these logs in your own infrastructure or mask error message contents. + +--- + +## Modifying compute log storage + +Dagster's compute logs are handled by the configured [`ComputeLogManager`](/api/python-api/internals#compute-log-manager). By default, Dagster+ utilizes the `CloudComputeLogManager` which stores logs in a Dagster+-managed Amazon S3 bucket, but you can customize this behavior to store logs in a destination of your choice. + +### Writing to your own S3 bucket + +If using the Kubernetes agent, you can instead forward logs to your own S3 bucket by using the [`S3ComputeLogManager`](/api/python-api/libraries/dagster-aws#dagster_aws.s3.S3ComputeLogManager). + +{/* You can configure the `S3ComputeLogManager` in your [`dagster.yaml` file](/dagster-plus/deployment/agents/customizing-configuration): */} +You can configure the `S3ComputeLogManager` in your [`dagster.yaml` file](/todo): + +```yaml +compute_logs: + module: dagster_aws.s3.compute_log_manager + class: S3ComputeLogManager + config: + show_url_only: true + bucket: your-compute-log-storage-bucket + region: your-bucket-region +``` + +If you are using Helm to deploy the Kubernetes agent, you can provide the following configuration in your `values.yaml` file: + +```yaml +computeLogs: + enabled: true + custom: + module: dagster_aws.s3.compute_log_manager + class: S3ComputeLogManager + config: + show_url_only: true + bucket: your-compute-log-storage-bucket + region: your-bucket-region +``` + +### Disabling compute log upload + +If your organization has its own logging solution which ingests `stdout` and `stderr` from your compute environment, you may want to disable compute log upload entirely. You can do this with the . + +{/* You can configure the `NoOpComputeLogManager` in your [`dagster.yaml` file](/dagster-plus/deployment/agents/customizing-configuration): */} +You can configure the `NoOpComputeLogManager` in your [`dagster.yaml` file](/todo): + +```yaml +compute_logs: + module: dagster.core.storage.noop_compute_log_manager + class: NoOpComputeLogManager +``` + +If you are using Helm to deploy the Kubernetes agent, use the `enabled` flag to disable compute log upload: + +```yaml +computeLogs: + enabled: false +``` + +### Other compute log storage options + +{/* For a full list of available compute log storage options, refer to the [Compute log storage docs](/deployment/dagster-instance#compute-log-storage). */} +For a full list of available compute log storage options, refer to the [Compute log storage docs](/todo). + +--- + +## Masking error messages + +By default, Dagster+ surfaces error messages from your code locations in the UI, including when runs fail, sensors or schedules throw an exception, or code locations fail to load. You can mask these error messages in the case that their contents are sensitive. + +To mask error messages in a Dagster+ Deployment, set the environment variable `DAGSTER_REDACT_USER_CODE_ERRORS` equal to `1` using the [**Environment variables** page](/dagster-plus/deployment/management/environment-variables/) in the UI: + + + +Once set, error messages from your code locations will be masked in the UI. A unique error ID will be generated, which you can use to look up the error message in your own logs. This error ID will appear in place of the error message in UI dialogs or in a run's event logs. + + + diff --git a/docs/docs-beta/static/images/dagster-cloud/logs-error-messages/configure-redact-env-var.png b/docs/docs-beta/static/images/dagster-cloud/logs-error-messages/configure-redact-env-var.png new file mode 100644 index 0000000000000..4efe4e0b654a9 Binary files /dev/null and b/docs/docs-beta/static/images/dagster-cloud/logs-error-messages/configure-redact-env-var.png differ diff --git a/docs/docs-beta/static/images/dagster-cloud/logs-error-messages/masked-err-message.png b/docs/docs-beta/static/images/dagster-cloud/logs-error-messages/masked-err-message.png new file mode 100644 index 0000000000000..d5e27296cc0bb Binary files /dev/null and b/docs/docs-beta/static/images/dagster-cloud/logs-error-messages/masked-err-message.png differ