Refresh "What's Dagster+" (#24007)

Aug 30th Lopp: 

This PR has been updated to incorporate a few changes (sorry for the
large size 😢 ):

- The Dagster+ getting started topology now has two docs: "what's
dagster+" and "getting started". This removes the dagster+ features doc.
- Getting started focuses on being an actionable guide, without trying
to re-describe a lot of what is in the product NUX. It relies fairly
heavily on the adding a code location guide instead of replicating the
content
- What's Dagster+ builds off of Jordan's original content to explain
briefly what Dagster+ is to a audience familiar with Dagster OSS, and
also adds the short list of Dagster+ features (hence the removal of the
features doc). This was also a natural place to put cross link out to
parts of the dagster.io site as well as other guides
- As a result, I decided to flatten the overall structure of the
dagster-plus docs folder. "What's dagster+" and "getting started" are
placed at the top level. Code requirements is moved to the deployment
section, allowing us to remove the entire getting-started directory.
- This PR also removes the "deployment types" doc in favor of the brief
description of Serverless vs Hybrid in the getting started / whats
dagster+ docs. Fixes the links as a result of this removal.
- Adds the hybrid doc landing page, which takes the rest of Jordan's
original content

---------

Co-authored-by: Sean Lopp <[email protected]>

docs/docs-beta/docs/dagster-plus/access/rbac/audit-logs.md

@@ -47,8 +47,8 @@ The **Filter** button near the top left of the page can be used to filter the li
 | Update sensor                  | A user toggles a sensor on or off                                                                             | The sensor name, code location, and cursor                                |
 | Update schedule                | A user toggles a schedule on or off                                                                           | The schedule name, code location, and cursor                              |
 | Update alert policy            | A user modifies an [alert policy](/dagster-plus/deployment/alerts/ui)                                         | The new configuration for the alert policy                                |
-| Create deployment              | A user creates a new [deployment](/dagster-plus/deployment-types)                                             | Whether the deployment is a branch deployment                             |
-| Delete deployment              | A user removes an existing [deployment](/dagster-plus/deployment-types)                                       | Whether the deployment is a branch deployment                             |
+| Create deployment              | A user creates a new deployment                                             | Whether the deployment is a branch deployment                             |
+| Delete deployment              | A user removes an existing deployment                                       | Whether the deployment is a branch deployment                             |
 | Create user token              | A user creates a new user token                                                                               |                                                                           |
 | Revoke user token              | A user revokes an existing user token                                                                         |                                                                           |
 | Change user permissions        | A user alters [permissions](/dagster-plus/access/rbac/user-roles-permissions) for another user                | The permission grant and targeted deployment                              |

docs/docs-beta/docs/dagster-plus/deployment/code-locations.md

@@ -5,7 +5,7 @@ displayed_sidebar: "dagsterPlus"
 
 # Code locations
 
-Separate code locations allow you to deploy different projects that still roll up into a single Dagster+ deployment with one global lineage graph. To add your first code location, follow the [getting started](/dagster-plus/getting-started) guide.
+Separate code locations allow you to deploy different projects that still roll up into a single Dagster+ deployment with one global lineage graph.
 
 This guide will cover three options for adding a new code location:
 - Adding a code location manually
@@ -15,7 +15,7 @@ This guide will cover three options for adding a new code location:
 <details>
 <summary>Prerequisites</summary>
 
-1. An existing Dagster project. Refer to the [recommended project structure](/tutorial/create-new-project) for more information.
+1. An existing Dagster project. Refer to the [recommended project structure](/tutorial/create-new-project) and [code requirements](/dagster-plus/deployment/code-requirements) pages for more information.
 
 2. Editor, Admin, or Organization Admin permissions in Dagster+.
 

docs/docs-beta/docs/dagster-plus/deployment/hybrid.md

@@ -4,4 +4,62 @@ displayed_sidebar: "dagsterPlus"
 sidebar_position: 2
 ---
 
-# Hybrid deployment
+# Hybrid deployment
+
+In a Dagster+ Hybrid deployment, the orchestration control plane is run by Dagster+ while your Dagster code is executed within your environment.
+
+[comment]: <> (TODO: Architecture diagram)
+
+## Get started
+
+To get started with a Hybrid deployment you'll need to:
+
+1. Create a [Dagster+ organization](https://dagster.cloud/signup)
+2. Install a Dagster+ Hybrid Agent
+3. [Add a code location](/dagster-plus/deployment/code-locations), typically using a Git repository and CI/CD
+
+## Dagster+ Hybrid agents
+
+The Dagster+ agent is a long-lived process that polls Dagster+'s API servers for new work.
+
+See the following guides for setting up an agent:
+ - [Kubernetes](/dagster-plus/deployment/hybrid/agents/kubernetes)
+ - [AWS ECS](/dagster-plus/deployment/hybrid/agents/amazon-ecs-new-vpc)
+ - [Docker](/dagster-plus/deployment/hybrid/agents/docker)
+ - [Locally](/dagster-plus/deployment/hybrid/agents/local)
+
+
+## What you'll see in your environment
+
+### Code location servers
+
+Dagster+ runs your Dagster projects through code locations. To get started, follow this guide for [adding a code location](/dagster-plus/deployment/code-locations).
+
+When you inform Dagster+ about a new code location, we enqueue instructions for your agent to launch a new code server. The agent uses your container image to launch a code server that interacts with your Dagster definitions. The agent will run one long-standing code server for each code location. Once the code server is running, the agent will send Dagster+ metadata about your Dagster definitions that Dagster+ uses to make orchestration decisions.
+
+
+### Runs
+
+Your definitions might include [automations](/guides/automation) that launch runs or materialize assets. Or your developers might launch runs directly with the web UI.
+
+When a run needs to be launched, Dagster+ enqueues instructions for your agent to launch a new run. The next time your agent polls Dagster+ for new work, it will see instructions about how to launch your run. It will delegate those instructions to your code server and your code server will launch a run - a new run will typically require its own container.
+
+Your agent will send Dagster+ metadata letting us know the run has been launched. Your run's container will also send Dagster+ metadata informing us of how the run is progressing. The Dagster+ backend services will monitor this stream of metadata to make additional orchestration decisions, monitor for failure, or send alerts.
+
+## Security
+
+Dagster+ hybrid relies on a shared security model.
+
+The Dagster+ control plane is SOC 2 Type II certified and follows best practices such as:
+- encrypting data at rest (AES 256) and in transit (TLS 1.2+)
+- highly available, with disaster recovery and backup strategies
+- only manages metadata such as pipeline names, execution status, and run duration
+
+The execution environment is managed by the customer:
+- your code never leaves your environment
+- all connections to databases, file systems, and other resources are made from your environment
+- the execution environment only requires egress access to Dagster+
+
+Common security considerations in Dagster+ hybrid include:
+- [disabling log forwarding](/todo)
+- [managing tokens](/todo)

docs/docs-beta/docs/dagster-plus/getting-started.md

@@ -3,4 +3,59 @@ title: "Getting started with Dagster+"
 displayed_sidebar: "dagsterPlus"
 ---
 
-# Get started with Dagster+
+# Get started with Dagster+
+
+First [create a Dagster+ organization](https://dagster.plus/signup). Note: you can sign up with:
+- a Google email address
+- a GitHub account
+- a one-time email link, great if you are using a corporate email. You can setup SSO after completing these steps.
+
+Next, pick your deployment type. Not sure?
+
+- [Dagster+ Serverless](/dagster-plus/deployment/serverless) is the easiest way to get started and is great for teams with limited DevOps support. In Dagster+ Serverless, your Dagster code is executed in Dagster+. You will need to be okay [giving Dagster+ the credentials](/dagster-plus/deployment/environment-variables) to connect to the tools you want to orchestrate.
+
+- [Dagster+ Hybrid](/dagster-plus/deployment/hybrid) is great for teams who want to orchestrate tools without giving Dagster+ direct access to your systems. Dagster+ Hybrid requires more DevOps support.
+
+The remaining steps depend on your deployment type.
+
+<Tabs>
+<TabItem value="serverless" label="Dagster+ Serverless">
+
+We recommend following the steps in Dagster+ to add a new project.
+
+[comment]: <> (TODO: Screenshot of Dagster+ serverless Nux)
+
+The Dagster+ on-boarding will guide you through:
+- creating a Git repository containing your Dagster code
+- setting up the necessary CI/CD actions to deploy that repository to Dagster+
+
+:::tip
+If you don't have any Dagster code yet, you will have the option to select an example quickstart project or import an existing dbt project
+:::
+
+See the guide on [adding code locations](/dagster-plus/deployment/code-locations) for details.
+</TabItem>
+
+<TabItem value="hybrid" label="Dagster+ Hybrid">
+
+## Install a Dagster+ Hybrid agent
+
+Follow [these guides](/dagster-plus/deployment/hybrid) for installing a Dagster+ Hybrid agent. Not sure which agent to pick? We recommend using the Dagster+ Kubernetes agent in most cases.
+
+
+## Setup CI/CD
+
+In most cases, your CI/CD process will be responsible for:
+- building your Dagster code into a Docker image
+- pushing your Docker image to a container registry you manage
+- notifying Dagster+ of the new or updated code
+
+Refer to the guide for [adding a code location](/dagster-plus/deployment/code-locations) for more detail.
+
+</TabItem>
+</Tabs>
+
+
+## Next steps
+
+Your Dagster+ account is automatically enrolled in a trial. You can [pick your plan type and enter your billing information](/dagster-plus/settings), or [contact the Dagster team](https://dagster.io/contact) if you need support or want to evaluate the Dagster+ Pro plan.

docs/docs-beta/docs/dagster-plus/whats-dagster-plus.md

@@ -0,0 +1,35 @@
+---
+title: "What's Dagster+?"
+displayed_sidebar: "dagsterPlus"
+sidebar_position: 1
+---
+
+# What's Dagster+?
+
+Dagster+ is a managed orchestration platform built on top of Dagster's open source engine. 
+
+Dagster+ is built to be the most performant, reliable, and cost effective way for data engineering teams to run Dagster in production. Dagster+ is also great for students, researchers, or individuals who want to explore Dagster with minimal overhead.
+
+Dagster+ comes in two flavors: a fully [Serverless](/dagster-plus/deployment/serverless) offering and a [Hybrid](/dagster-plus/deployment/hybrid) offering. In both cases, Dagster+ does the hard work of managing your data orchestration control plane. Compared to a [Dagster open source deployment](/guides/deployment/), Dagster+ manages:
+ - Dagster's web UI at https://dagster.plus
+ - Metadata stores for data cataloging and cost insights
+ - Backend services for orchestration, alerting, and more
+
+Dagster+ Serverless is fully managed and your Dagster code executes in our environment. In Dagster+ Hybrid, you run an execution environment that connects to the Dagster+ control plane. 
+
+In addition to managed infrastructure, Dagster+ also adds core capabilities on top of Dagster open source to enable teams building data platforms:
+
+- [Insights](/dagster-plus/insights), a powerful tool for identifying trends in your data platform overtime, optimizing cost, and answering questions like "Why does it feel like our pipelines are taking longer this month?".
+- [Alerts](/dagster-plus/deployment/alerts) to a variety of services like Slack, PagerDuty, and email to notify your team of failed runs, data quality issues, and violated SLAs.
+- Authentication, [Role Based Access Control](/dagster-plus/access/rbac), and [Audit Logs](/dagster-plus/access/rbac/audit-logs) which help teams implement data mesh strategies while remaining compliant.
+- [Data Catalog](/dagster-plus/data-catalog), a powerful search-first experience that builds off of Dagster's best-in-class lineage graph to include searching for assets, metadata, column lineage, and more.
+- [Branch Deployments](/dagster-plus/deployment/branch-deployments)
+
+Ready to [get started](/dagster-plus/getting-started)?
+
+## Other resources
+- Learn more about Dagster+ [pricing and plan types](https://dagster.io/pricing) or [contact the Dagster team](https://dagster.io/contact)
+- Dagster+ includes support, [click here](https://dagster.io/support) to learn more.
+- Dagster+ is HIPAA compliant, SOC 2 Type II certified, and meets GDPR requirements. Learn more about Dagster+[ security](https://dagster.io/security).
+- Migrate [from a Dagster open source deployment to Dagster+](/guides/deployment/self-hosted-to-dagster-plus)
+- Dagster+ [status page](https://dagstercloud.statuspage.io/)

docs/docs-beta/docs/guides/deployment/dagster-plus.md

@@ -4,3 +4,5 @@ sidebar_position: 40
 ---
 
 # Deploy to Dagster+
+
+See the [Dagster+ getting started](/dagster-plus/getting-started) guide.

docs/docs-beta/sidebars.ts

@@ -254,11 +254,11 @@ const sidebars: SidebarsConfig = {
       items: [
         {
           type: 'doc',
-          id: 'dagster-plus/getting-started/whats-dagster-plus',
+          id: 'dagster-plus/whats-dagster-plus',
         },
         {
           type: 'doc',
-          id: 'dagster-plus/getting-started/code-requirements',
+          id: 'dagster-plus/getting-started',
         },
       ],
     },
@@ -267,11 +267,6 @@ const sidebars: SidebarsConfig = {
       label: 'Features',
       collapsible: false,
       items: [
-        {
-          type: 'doc',
-          label: 'Deployment types',
-          id: 'dagster-plus/deployment-types',
-        },
         {
           type: 'category',
           label: 'Insights',
@@ -340,10 +335,6 @@ const sidebars: SidebarsConfig = {
             },
           ],
         },
-        {
-          type: 'doc',
-          id: 'dagster-plus/multi-tenancy',
-        },
         {
           type: 'category',
           label: 'Authentication & access control',
@@ -424,11 +415,6 @@ const sidebars: SidebarsConfig = {
             id: 'dagster-plus/deployment/hybrid',
           },
           items: [
-            {
-              type: 'doc',
-              label: 'Architecture',
-              id: 'dagster-plus/deployment/hybrid/architecture',
-            },
             {
               type: 'doc',
               label: 'Tokens',

docs/vale/styles/config/vocabularies/Dagster/accept.txt

@@ -79,3 +79,14 @@ uncomment
 vCPU
 vCPUs
 we have
+
+
+SLA
+SLAs
+performant
+SOC
+GDPR
+HIPAA
+IAM
+ECS
+AWS