pr comments

dagster-io · Aug 27, 2024 · 74d85dc · 74d85dc
1 parent 646a985
commit 74d85dc
Show file tree

Hide file tree

Showing 5 changed files with 42 additions and 53 deletions.
diff --git a/docs/docs-beta/docs/guides/external-systems/apis.md b/docs/docs-beta/docs/guides/external-systems/apis.md
@@ -3,14 +3,14 @@ title: Connecting to APIs
 sidebar_position: 20
 ---
 
-This guide describes how to connect to and interact with APIs in dagster. In this guide you will use dagster Resources to connect to an external API. Using a Resource allows you to standardize how you connect to an external API across your project and use configuration to customize your connections.
+When building a data pipeline, you'll likely need to connect to a number of external APIs, each with its own specific configuration and behavior. This guide demonstrates how to standardize your API connections and customize their configuration using Dagster resources.
 
 
 ## What you'll learn
 
-- How to write a dagster Resource to connect to an API
-- How to use that Resource in an asset
-- How to configure a Resource
+- How to connect to an API using a Dagster resource
+- How to use that resource in an asset
+- How to configure a resource
 - How to source configuration values from environment variables
 
 <details>
@@ -19,53 +19,60 @@ This guide describes how to connect to and interact with APIs in dagster. In thi
 To follow the steps in this guide, you'll need:
 
 - Familiarity with [Asset definitions](/concepts/assets)
-- Familiarity with [Resources](/concepts/resources)
-- Install the `requests` library: `pip install requests`
+- Familiarity with [resources](/concepts/resources)
+- Install the `requests` library:
+    ```bash
+    pip install requests
+    ```
 
 </details>
 
-## Step 1: Write a Resource to connect to an API
+## Step 1: Write a resource to connect to an API
 
-Here is a minimal Resource for connecting to the an API that returns the sunrise time. In this example, the request URL has been hardcoded to query for data at San Francisco International Airport.
+This example fetches the sunrise time from a REST API for a given location. Begin by defining a Dagster resource with a method to return the sunrise time for a location. In the first version of this resource, the location will be hardcoded to San Francisco International Airport.
 
-<CodeExample filePath="guides/external-systems/apis/minimal_resource.py" language="python" title="Resource to connect to Sun API" />
 
+<CodeExample filePath="guides/external-systems/apis/minimal_resource.py" language="python" title="Resource to connect to the Sunrise API" />
 
-## Step 2: Use the Resource in an asset
 
-To use the Resource written in Step 1, you can provide it to an asset like this:
+## Step 2: Use the resource in an asset
 
-<CodeExample filePath="guides/external-systems/apis/use_minimal_resource_in_asset.py" language="python" title="Use the SFOSunResource in an asset" />
+To use the resource written in Step 1, you can provide it as a parameter to an asset:
 
-When you materialize `sfo_sunrise`, dagster will provide an initialized `SFOSunResource` to the `sun_resource` parameter.
+<CodeExample filePath="guides/external-systems/apis/use_minimal_resource_in_asset.py" language="python" title="Use the SunResource in an asset" />
 
+When you materialize `sfo_sunrise`, Dagster will provide an initialized `SunResource` to the `sun_resource` parameter.
 
-## Step 3: Configure your Resource
-Many APIs have configuration you can set to customize your usage. Here is an updated version of the Resource from Step 1 with configuration to allow for setting the query location:
 
-<CodeExample filePath="guides/external-systems/apis/configurable_resource.py" language="python" title="Configurable Resource to connect to Sun API" />
+## Step 3: Configure your resource
+Many APIs have configuration you can set to customize your usage. Here is an updated version of the resource from Step 1 with configuration to allow for setting the query location:
 
-## Step 4: Use the configurable Resource in an asset
+<CodeExample filePath="guides/external-systems/apis/use_configurable_resource_in_asset.py" language="python" title="Use the configurable SunResource in an asset" />
 
-The configurable Resource written in Step 3 can be provided to an asset exactly as before. When the Resource is initialized, you can pass values for each of the configuration options.
+The configurable resource can be provided to an asset exactly as before. When the resource is initialized, you can pass values for each of the configuration options.
 
-<CodeExample filePath="guides/external-systems/apis/use_configurable_resource_in_asset.py" language="python" title="Use the configurable SunResource in an asset" />
+When you materialize `sfo_sunrise`, Dagster will provide a `SunResource` initialized with the configuration values to the `sun_resource` parameter.
 
-When you materialize `sfo_sunrise`, dagster will provide a `SunResource` initialized with the provided configuration values to the `sun_resource` parameter.
 
+## Step 4: Sourcing configuration values from environment variables
+Resources can also be configured with environment variables, specifically values referenced using the `EnvVar` class.
 
-## Step 5: Sourcing configuration values from environment variables
+In this example, there is a new `home_sunrise` asset. Rather than hardcoding the location of your home, you can set it in environment variables, and configure the `SunResource` by reading those values:
 
-You can configure your Resource using values that are stored in environment variables using the `EnvVar` class. In this example, there is a new `home_sunrise` asset. Rather than hardcoding the location of your home, you can set it in environment variables, and configure the `SunResource` by reading those values:
+<CodeExample filePath="guides/external-systems/apis/env_var_configuration.py" language="python" title="Configure the resource with values from environment variables" />
 
-<CodeExample filePath="guides/external-systems/apis/env_var_configuration.py" language="python" title="Configure the Resource with values from environment variables" />
+When you materialize `home_sunrise`, Dagster will read the values set for the `HOME_LATITUDE`, `HOME_LONGITUDE`, and `HOME_TIMZONE` environment variables and initialize a `SunResource` with those values.
 
-When you materialize `home_sunrise`, dagster will read the values set for the `HOME_LATITUDE`, `HOME_LONGITUDE, and `HOME_TIMZONE` environment variables and initialize a `SunResource` with those values. That initialized `SunResource` will be provided to the `sun_resource` parameter. Note that the values are read **at materialization time**, so the environment variables must be set where the code is executing. 
+The initialized `SunResource` will be provided to the `sun_resource` parameter.
 
+:::note
+Environment variables are read **at materialization time**, so they must be set where the code is executing.
+:::
 
 
 ## Next steps
 
 - [Authenticate to a resource](/guides/external-systems/authentication.md)
-- [Use different Resources in different execution environments](/todo)
-- Learn what [dagster-provided Resources](/todo) are available to use
+- [Use different resources in different execution environments](/todo)
+- [Set environment variables in Dagster+](/todo)
+- Learn what [Dagster-provided resources](/todo) are available to use
diff --git a/...cs_beta_snippets/docs_beta_snippets/guides/external-systems/apis/configurable_resource.py b/...cs_beta_snippets/docs_beta_snippets/guides/external-systems/apis/configurable_resource.py
diff --git a/...es/docs_beta_snippets/docs_beta_snippets/guides/external-systems/apis/minimal_resource.py b/...es/docs_beta_snippets/docs_beta_snippets/guides/external-systems/apis/minimal_resource.py
@@ -3,7 +3,7 @@
 import dagster as dg
 
 
-class SFOSunResource(dg.ConfigurableResource):
+class SunResource(dg.ConfigurableResource):
     @property
     def query_string(self) -> str:
         latittude = "37.615223"

diff --git a/...ets/docs_beta_snippets/guides/external-systems/apis/use_configurable_resource_in_asset.py b/...ets/docs_beta_snippets/guides/external-systems/apis/use_configurable_resource_in_asset.py
@@ -4,26 +4,29 @@
 
 
 class SunResource(dg.ConfigurableResource):
+    # highlight-start
     latitude: str
     longitude: str
     time_zone: str
 
     @property
     def query_string(self) -> str:
-        return f"https://api.sunrise-sunset.org/json?lat={self.latitude}&lng={self.longitude}&date=today&tzid={self.time_zone}"
+        return f"https://api.sunrise-sunset.org/json?lat={self.latittude}&lng={self.longitude}&date=today&tzid={self.time_zone}"
+
+    # highlight-end
 
     def sunrise(self) -> str:
         data = requests.get(self.query_string, timeout=5).json()
         return data["results"]["sunrise"]
 
 
-# highlight-start
 @dg.asset
 def sfo_sunrise(context: dg.AssetExecutionContext, sun_resource: SunResource) -> None:
     sunrise = sun_resource.sunrise()
     context.log.info(f"Sunrise in San Francisco is at {sunrise}.")
 
 
+# highlight-start
 defs = dg.Definitions(
     assets=[sfo_sunrise],
     resources={

diff --git a/...snippets/docs_beta_snippets/guides/external-systems/apis/use_minimal_resource_in_asset.py b/...snippets/docs_beta_snippets/guides/external-systems/apis/use_minimal_resource_in_asset.py
@@ -3,7 +3,7 @@
 import dagster as dg
 
 
-class SFOSunResource(dg.ConfigurableResource):
+class SunResource(dg.ConfigurableResource):
     @property
     def query_string(self) -> str:
         latittude = "37.615223"
@@ -18,15 +18,11 @@ def sunrise(self) -> str:
 
 # highlight-start
 @dg.asset
-def sfo_sunrise(
-    context: dg.AssetExecutionContext, sun_resource: SFOSunResource
-) -> None:
+def sfo_sunrise(context: dg.AssetExecutionContext, sun_resource: SunResource) -> None:
     sunrise = sun_resource.sunrise()
     context.log.info(f"Sunrise in San Francisco is at {sunrise}.")
 
 
-defs = dg.Definitions(
-    assets=[sfo_sunrise], resources={"sun_resource": SFOSunResource()}
-)
+defs = dg.Definitions(assets=[sfo_sunrise], resources={"sun_resource": SunResource()})
 
 # highlight-end