Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC - [computelogmanager] add fields to pass uri, path or shell cmd #26723

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

mlarose
Copy link
Contributor

@mlarose mlarose commented Dec 26, 2024

Summary & Motivation

The Azure Blob Storage Container do not provide direct urls to objects on private storage accounts, even if the user is authenticated.

This PR provide new fields on the captured logs entry that such ComputeLogManager could populate to improve the user experience by providing a path or uri and/or a shell command to access those logs.

How I Tested These Changes

  • tested locally with a modified AzureBlobComputeLogManager
  • TODO:
  • add automated tests
  • test with local host cloud

Changelog

Insert changelog entry or delete this section.

Copy link
Contributor Author

mlarose commented Dec 26, 2024

This stack of pull requests is managed by Graphite. Learn more about stacking.

@mlarose mlarose changed the title [computelogmanager] add fields to pass uri, path or shell cmd RFC - [computelogmanager] add fields to pass uri, path or shell cmd Dec 26, 2024
@mlarose mlarose marked this pull request as ready for review December 26, 2024 16:14
external_stdout_url=log_context.external_stdout_url,
external_stderr_url=log_context.external_stderr_url,
stdout_shell_cmd=log_context.stdout_shell_cmd,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is adding fields to a LOGS_CAPTURED event payload problematic?

{externalUrl}
</a>
<Icon name="open_in_new" color={Colors.tooltipText()} size={20} style={{marginTop: 2}} />
{externalUrl ? (
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will definitely need css+html rework, i was just passing information and making the click to copy work.

when thinking about this ui, should we provide even more information, such as additional text about to use the shell command (ie, requiring a preconfigured azure cli, for example) ? I stopped short of this because I was already concerned about increasing the event size, but maybe I could add a 'log type' field that the UI could use to render that additional information without producing it from the ComputeLogManager

Copy link

Deploy preview for dagit-core-storybook ready!

✅ Preview
https://dagit-core-storybook-qhuups8a8-elementl.vercel.app
https://mlarose-azure-blob-compute-log-manager-uri.core-storybook.dagster-docs.io

Built with commit ce0b204.
This pull request is being automatically deployed with vercel-action

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant