[dagster-embedded-elt] Expose dlt and sling extras

[deepyaman]

Summary & Motivation

As a user, I don't unnecessarily want to install Sling if I'm only using dlt and vice versa.

How I Tested These Changes

Changelog

dlt and Sling are now extra requirements of the dagster-embedded-elt package; to include them, install dagster-embedded-elt[dlt] or dagster-embedded-elt[sling], respectively.

[github-actions]

Deploy preview for dagster-docs-beta ready!

Preview available at https://dagster-docs-beta-b7wh6wgpd-elementl.vercel.app

Direct link to changed pages:

• https://dagster-docs-beta-b7wh6wgpd-elementl.vercel.app/integrations/libraries/dlt
• https://dagster-docs-beta-b7wh6wgpd-elementl.vercel.app/integrations/libraries/sling

[github-actions]

Deploy preview for dagster-docs ready!

Preview available at https://dagster-docs-63xuyaycb-elementl.vercel.app
https://deepyaman-build-embedded-elt-extras.dagster.dagster-docs.io

Direct link to changed pages:

[OwenKephart]

@cmpadden @PedramNavid do either of you recall why we didn't do it this way to begin with? I remember there was some discussion about having embedded-elt be its own package vs. having dagster-sling and dagster-dlt, wondering if there's some context missing there

[PedramNavid]

Can’t think of any reason why we wouldn’t do this. Will this be a breaking change?

[OwenKephart]

discussed offline: this would unfortunately be a breaking change

[zilto]

As a user, I was about to open a issue for the same reason.

Inconvenience

• The pypi target dagster-embbed-elt doesn't follow the integration convention and I had to go through the docs to find it ("embbed elt" could be obscure terms for users)
• I don't want to install sling or depend on it. The Python install seems small, but it seems to require a binary that's about 50Mb, which might matter on AWS lambdas and such.
• The Python imports dagster_embedded_elt may be more obvious with dagster_sling and dagster_dlt

Potential solution

There seems to be two-fold for backwards compatibility:

• pypi: create pypi targets dagster-dlt and dagster-sling while keeping dagster-embbed-elt
• python modules: Could have a dagster-dlt namespace that imports dagster_embbed_elt.dlt

[OwenKephart]

Hi @zilto ! Thanks for weighing in here -- we're currently planning on segmenting these libraries out essentially as you describe, although we'll likely just port the code for each integration into the new libraries and keep dagster-embedded-elt as a thin backcompat layer with dependencies on dagster-sling and dagster-dlt (rather than vice-versa)

[deepyaman]

Closing this issue and creating #26523 to track the effort to split out these libraries.