Skip to content

Commit

Permalink
first pass
Browse files Browse the repository at this point in the history
  • Loading branch information
jamiedemaria committed Aug 26, 2024
1 parent 5d33620 commit 50dfe4a
Show file tree
Hide file tree
Showing 3 changed files with 83 additions and 1 deletion.
39 changes: 38 additions & 1 deletion docs/docs-beta/docs/guides/external-systems/apis.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,41 @@
---
title: Connecting to APIs
sidebar_position: 20
---
---

This guide describes how to connect to and interact with APIs in dagster.


## What you'll learn

- How to write a dagster Resource to connect to an API
- How to use that Resource in an asset

<details>
<summary>Prerequisites</summary>

To follow the steps in this guide, you'll need:

- Familiarity with [Asset definitions](/concepts/assets)
- Familiarity with [Resources](/concepts/resources)
- Install the `requests` library: `pip install requests`

</details>

## Step 1: Write a Resource to connect to an API

Here is a minimal Resource for connecting to the an API that returns data about the sunrise and sunset times. In this example, the request URL has been hardcoded to query for data at San Francisco International Airport.

<CodeExample filePath="guides/external-systems/apis/minimal_resource.py" language="python" title="Simple Resource to connect to Sun API" />


## Step 2: Use the Resource in an asset

To use the Resource written in step 1, you can provide it to an asset like this:

<CodeExample filePath="guides/external-systems/apis/use_resource_in_asset.py" language="python" title="Use the XYZResource in an asset" />

## Next steps

- [Customize resources with configuration](/todo)
- Learn what [dagster-provided Resources](/todo) are available to use
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
import requests

import dagster as dg


class SFOSunResource(dg.ConfigurableResource):
@property
def query_string(self) -> str:
latittude = 37.615223
longitude = -122.389977
tzone = "America/Los_Angeles"
return f"https://api.sunrise-sunset.org/json?lat={latittude}&lng={longitude}&date=today&tzid={tzone}"

def sunrise(self) -> str:
data = requests.get(self.query_string, timeout=5).json()
return data["results"]["sunrise"]
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
import requests

import dagster as dg


class SFOSunResource(dg.ConfigurableResource):
@property
def query_string(self) -> str:
latittude = 37.615223
longitude = -122.389977
tzone = "America/Los_Angeles"
return f"https://api.sunrise-sunset.org/json?lat={latittude}&lng={longitude}&date=today&tzid={tzone}"

def sunrise(self) -> str:
data = requests.get(self.query_string, timeout=5).json()
return data["results"]["sunrise"]


@dg.asset
def sfo_sunrise(
context: dg.AssetExecutionContext, sun_resource: SFOSunResource
) -> None:
sunrise = sun_resource.sunrise()
context.log.info(f"Sunrise in San Francisco is at {sunrise}.")


defs = dg.Definitions(
assets=[sfo_sunrise], resources={"sun_resource": SFOSunResource()}
)

0 comments on commit 50dfe4a

Please sign in to comment.