Skip to content

Commit

Permalink
move example to advanced
Browse files Browse the repository at this point in the history
  • Loading branch information
AstrakhantsevaAA committed Jan 13, 2025
1 parent 9044bfc commit b900539
Show file tree
Hide file tree
Showing 2 changed files with 26 additions and 29 deletions.
27 changes: 26 additions & 1 deletion docs/website/docs/general-usage/credentials/advanced.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,32 @@ dlt.secrets["destination.postgres.credentials"] = BaseHook.get_connection('postg

This will mock the TOML provider to desired values.

### Example
## Configuring destination credentials in code

If you need to manage destination credentials programmatically, such as retrieving them from environment variables, you can define them directly in your pipeline code.

The following example demonstrates how to use [GcpServiceAccountCredentials](complex_types#gcp-credentials) to set up a pipeline with a BigQuery destination.

```py
import os

import dlt
from dlt.sources.credentials import GcpServiceAccountCredentials
from dlt.destinations import bigquery

# Retrieve credentials from the environment variable
creds_dict = os.getenv('BIGQUERY_CREDENTIALS')

# Create credentials instance and parse them from a native representation
gcp_credentials = GcpServiceAccountCredentials()
gcp_credentials.parse_native_representation(creds_dict)

# Pass the credentials to the BigQuery destination
pipeline = dlt.pipeline(destination=bigquery(credentials=gcp_credentials))
pipeline.run([{"key1": "value1"}], table_name="temp")
```

## Example

In the example below, the `google_sheets` source function is used to read selected tabs from Google Sheets.
It takes several arguments that specify the spreadsheet, the tab names, and the Google credentials to be used when extracting data.
Expand Down
28 changes: 0 additions & 28 deletions docs/website/docs/walkthroughs/add_credentials.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,34 +74,6 @@ DESTINATION__BIGQUERY__CREDENTIALS__PRIVATE_KEY
DESTINATION__BIGQUERY__CREDENTIALS__CLIENT_EMAIL
DESTINATION__BIGQUERY__LOCATION
```
### Configuring destination credentials in code

If you need to manage destination credentials programmatically, such as retrieving them from environment variables, you can define them directly in your pipeline code.

The following example demonstrates how to use `GcpServiceAccountCredentials` to set up a pipeline with a BigQuery destination.

```py
import os

import dlt
from dlt.sources.credentials import GcpServiceAccountCredentials
from dlt.destinations import bigquery

# Retrieve credentials from the environment variable
creds_dict = os.getenv('BIGQUERY_CREDENTIALS')

# Create credentials instance and parse them from a native representation
gcp_credentials = GcpServiceAccountCredentials()
gcp_credentials.parse_native_representation(creds_dict)

# Pass the credentials to the BigQuery destination
pipeline = dlt.pipeline(destination=bigquery(credentials=gcp_credentials))
pipeline.run([{"key1": "value1"}], table_name="temp")
```

In the example above, we retrieve service account credentials using `os.getenv`, parse them, and attach them to the pipeline's destination.

For more details on GCP credentials, see [our documentation.](../general-usage/credentials/complex_types#gcp-credentials)

## Retrieving credentials from Google Cloud Secret Manager

Expand Down

0 comments on commit b900539

Please sign in to comment.