Skip to content

Commit

Permalink
legacy docs: update integration landing page (#24524)
Browse files Browse the repository at this point in the history
## Summary & Motivation

This is a long overdue PR that:

- Removes confusing contents from integration docs (legacy docs) and
redirects to the most relevant page:
  | integration | old docs | changes |
  | --- | --- | --- |
| great expectations | /integrations/great-expectations | removed and
redirected to
https://dagster.io/blog/ensuring-data-quality-with-dagster-and-great-expectations
|
| airbyte | /guides/dagster/airbyte-ingestion-as-code | removed and
redirected to /integrations/airbyte |
| databricks | /integrations/databricks | removed and redirected to
Pipes /concepts/dagster-pipes/databricks |

- Removes libraries that are not considered as integrations. Mainly two
cohorts:
- OSS deployment related (most won't apply to Dagster+ users; and direct
deployments to their own doc sections):
    - Celery
    - Celery & Docker
    - Dask
    - Docker
    - MySQL
    - PostgreSQL
  - quality too low
    - Papaertrial
    - MLflow
- Adds good integrations to the integration landing page:
  - AWS ECS
  - AWS Lambda
  - AWS Glue
  - Kubernetes
  - Shell / Bash
- Adds more community-supported integrations to the integration landing
page:
  - Census (moved from core)
  - Cube
  - Delta Lake
  - SDF
  - Open Metadata
NOTE: we have meltano and hex but they are both op-based integrations,
so i tend to not include them until we or have the partner update them.


## How I Tested These Changes

https://09-16-legacy-docs-update-integration-landing-page.dagster.dagster-docs.io/integrations

## Changelog

`NOCHANGELOG`
  • Loading branch information
yuhan authored Sep 16, 2024
1 parent aed17e5 commit 8551263
Show file tree
Hide file tree
Showing 6 changed files with 85 additions and 499 deletions.
10 changes: 1 addition & 9 deletions docs/content/_navigation.json
Original file line number Diff line number Diff line change
Expand Up @@ -889,10 +889,6 @@
{
"title": "Airbyte Cloud & Dagster",
"path": "/integrations/airbyte-cloud"
},
{
"title": "Ingestion as code",
"path": "/guides/dagster/airbyte-ingestion-as-code"
}
]
},
Expand All @@ -902,7 +898,7 @@
},
{
"title": "Databricks",
"path": "/integrations/databricks"
"path": "/concepts/dagster-pipes/databricks"
},
{
"title": "dbt",
Expand Down Expand Up @@ -1004,10 +1000,6 @@
"title": "Fivetran",
"path": "/integrations/fivetran"
},
{
"title": "Great Expectations",
"path": "/integrations/great-expectations"
},
{
"title": "Google BigQuery",
"path": "/integrations/bigquery",
Expand Down
223 changes: 0 additions & 223 deletions docs/content/guides/dagster/airbyte-ingestion-as-code.mdx

This file was deleted.

Loading

1 comment on commit 8551263

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deploy preview for dagster-docs ready!

✅ Preview
https://dagster-docs-ps9lxdtkv-elementl.vercel.app
https://master.dagster.dagster-docs.io

Built with commit 8551263.
This pull request is being automatically deployed with vercel-action

Please sign in to comment.