Skip to content

Commit

Permalink
updated images for v51, added data warehouse tutorial, nit changes
Browse files Browse the repository at this point in the history
  • Loading branch information
royendo committed Nov 21, 2024
1 parent 1c97974 commit 5572d98
Show file tree
Hide file tree
Showing 23 changed files with 231 additions and 141 deletions.
21 changes: 14 additions & 7 deletions docs/docs/build/dashboards/dashboards.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,21 +32,15 @@ metrics_view: my_metricsview
dimensions: '*' #can use regex
measures: '*' #can use regex

theme: #your default theme

time_ranges: #was available_time_ranges, list the time of available time ranges that can be selected in your dashboard
time_zones: #was available_time_zones, list the time zones that are selectable in the dashboard

defaults: #define all the defaults within here, was default_* in previous dashboard YAML
dimensions:
measures:
time_range:
comparison_mode:
comparison_dimension:

...
security:
access: #only dashboard access can be defined here, other security policies must be set on the metrics view

```
Expand All @@ -62,6 +56,19 @@ Once a dashboard is ready to preview, before [deploying to Rill Cloud](/deploy/d
![preview](/img/build/dashboard/preview-dashboard.png)
### Clickable Dimension Links
Adding an additional parameter to your dimension in the [metrics view](/build/metrics-view/) can allow for clickable links directly from the dashboard.
```yaml
dimensions:
- label: Company Url
column: Company URL
uri: true #if already set to the URL, also accepts SQL expressions
```
![url-click](/img/build/dashboard/clickable-dimension.png)
### Multi-editor and external IDE support
Rill Developer is meant to be developer friendly and has been built around the idea of keystroke-by-keystroke feedback when modeling your data, allowing live interactivity and a real-time feedback loop to iterate quickly (or make adjustments as necessary) with your models and dashboards. Additionally, Rill Developer has support for the concept of "hot reloading", which means that you can keep two windows of Rill open at the same time and/or use a preferred editor of choice, such as VSCode, side-by-side with the dashboard that you're actively developing!
Expand Down
4 changes: 4 additions & 0 deletions docs/docs/build/incremental-models/incremental-models.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,10 @@ sql: |

Once state is defined in an incremental model, its value can be used as a variable in your SQL statement. In the above example, the state returns the most recent `date` value from `TABLE` and adds an additional day. Then, the SQL statement will run based on the WHERE clause.

:::tip
You can verify the current value of your state in the left hand panel under Incremental Processing.
:::


In the above example, we are using patitions defined in DuckDB to define a range of days to use in the Snowflake query. The data will be written to a temp-data folder in S3 and written to ClickHouse after. Once completed, the data in temp-data will be cleared.

Expand Down
2 changes: 1 addition & 1 deletion docs/docs/build/metrics-view/metrics-view.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ For more information about available metrics view properties, feel free to check
:::


### Multi-editor and external IDE support
## Multi-editor and external IDE support

Rill Developer is meant to be developer friendly and has been built around the idea of keystroke-by-keystroke feedback when modeling your data, allowing live interactivity and a real-time feedback loop to iterate quickly (or make adjustments as necessary) with your models and dashboards. Additionally, Rill Developer has support for the concept of "hot reloading", which means that you can keep two windows of Rill open at the same time and/or use a preferred editor of choice, such as VSCode, side-by-side with the dashboard that you're actively developing!

Expand Down
6 changes: 0 additions & 6 deletions docs/docs/deploy/deploy-dashboard/trial-accounts.md

This file was deleted.

2 changes: 1 addition & 1 deletion docs/docs/latest-changes/v50-dashboard-changes.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ With the split of metrics view, dashboard configurations experienced an overhaul

## How to migrate your current Dashboards

### version 0.49 -> version 0.50
### version 0.49 -> version 0.5X

Due to the [separation of dashboards to metrics layer and dashboards](/concepts/metrics-layer), you will need to review your current dashboards and make the following changes (note: Legacy dashboards will continue to function.):

Expand Down
4 changes: 4 additions & 0 deletions docs/docs/manage/account-management/_category_.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
position: 90
label: Account Management
collapsible: true
collapsed: true
30 changes: 30 additions & 0 deletions docs/docs/manage/account-management/billing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
---
title: "Billing"
description: How Billing works for non-enterprise accounts
sidebar_label: "Billing Information"
sidebar_position: 00
---

## Overview

Billing cycles begin on the first of the every month (12:00am UTC). If you start your plan mid-monthm, your first month will be pro-rated accordingly.


### How does it work?
Rill Data does not use a user-based license system. Instead, we calculate your data usage, after ingestion, and calculate the pricing based on this. For more information on pricing, see our [pricing page](https://www.rilldata.com/pricing).

## *Plan Details*

### Team Plan



### Trial Plan

Get started with Rill Cloud with our 30 day free trial! Upon deployment of your first project, your trial will automatically start. See your Usage and Billing information in the Settings page.

![deploy](/img/manage/billing/deploy-project.png)
## Managing Payment Information


### Accepted Payment methods
4 changes: 4 additions & 0 deletions docs/docs/manage/project-management.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,10 @@ A project is a single deployed instance from Rill Developer (or what we refer to
### Checking deployment status
After deploying to Rill Cloud, if your projects are not quite ready to view yet, you will be navigating to the status page. Here you will be able to see your component's status and if there are any issues with loading or parsing.

:::tip Refresh all source and models
You can select the `Refresh all sources and models` in the Status page or run a full project refresh.
:::

![img](/img/manage/project-management/status.png)

### Managing Project settings
Expand Down
88 changes: 86 additions & 2 deletions docs/docs/reference/project-files/alerts.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,90 @@ title: Alert YAML
sidebar_label: Alert YAML
sidebar_position: 60
---
https://github.com/rilldata/rill/blob/d39df9f11a8300095e6ea247b1b5c2d2fd6b4ecb/runtime/compilers/rillv1/parse_alert.go#L18

Wait for confirmation on Product to add this YAML
Along with alertings at the dashboard level and can be created via the UI, there might be more extensive alerting that you might want to develop and can be done so the an alert.yaml. When creating an alert via a YAML file, you'll see this denoted in the UI as `Created through code`.

**`type`** — Refers to the resource type and must be `alert` _(required)_.

**`title`** — Refers to the display name for the metrics view [deprecated, use `display_name`] _(required)_.

**`display_name`** - Refers to the display name for the metrics view _(required)_.

**`refresh`** - Specifies the refresh schedule that Rill should follow to re-ingest and update the underlying source data _(optional)_.
- **`cron`** - a cron schedule expression, which should be encapsulated in single quotes, e.g. `'* * * * *'` _(optional)_.
- **`every`** - a Go duration string, such as `24h` ([docs](https://pkg.go.dev/time#ParseDuration)) _(optional)_.
- **`disable`** - boolean, completely disable the resource, without deleting it _(optional)_.
- **`ref_update`** -: boolean, don't refresh when a dependency is refreshed _(optional)_.
```
refresh:
cron: "0 8 * * *"
```

**`intervals`** - define the interval of the alert to check _(required)_.
- **`duration`** - a valid ISO8601 duration to define the interval duration. _(required)_.
- **`limit`** - maximum number of intervals to check for on invocation _(optional)_.
- **`check_unclosed`** - boolean, whether unclosed intervals should be checked _(optional)_.

```yaml
intervals:
duration: 'P3D'
# limit: 5
# check_unclosed: true
```

**`timeout`** - define the timeout of the alert in seconds _(optional)_.

**`data`** - define the alert constraints using various parameters _(required)_.
- **`connector`** - if running a SQL query or using `glob`, will need to define what connector to use _(optional)_.
- **`sql`** - raw SQL query to run against existing tables and views in your project _(optional)_.
- **`metrics_sql`** - a SQL query against a metrics view in your project _(optional)_.
- **`api`** - name of existing custom API in your project _(optional)_.
- **`args`** - used with `api` to define args to be passed to the API _(optional)_.
- **`glob`** - define the path in your connector _(optional)_.
- **`resource_status`** -
- **`where_error`** - boolean, if the returning data alert constraints returns true or false _(required)_.

```yaml
#Alert will trigger if any of the project's resources return with a reconile error.
data:
resource_status:
where_error: true
```
**`on_recover`** - boolean, send alert on recovery, defaults to false _(optional)_.

**`on_fail`** - boolean, send alert of failure, defaults to true _(optional)_.

**`on_error`** - boolean, send the alert on error, defaults to false _(optional)_.

**`renotify`** - boolean, enable to disable renotifcation of alert, defaults to false _(optional)_.

**`renotify_after`** - define the renotification of the alert in seconds, equiavalent to snooze duration in UI, defaults to 'Off' _(optional)_.

```yaml
on_recover: true
on_fail: true
on_error: true
renotify: true
# renotify_after: 360m
```

**`notify`** - define where to notify the user of the defined alert _(required)_.
- **`email`** -
- **`recipients`** - an array of emails to send the alert to _(optional)_.
- **`slack`** -
- **`users`** - an array of Slack users to send the alert notification to _(optional)_.
- **`channels`** - an array of Slack channels to send the alert notification to _(optional)_.
- **`webhooks`** - an array of webhooks to send the alert notification to _(optional)_.

```yaml
# Send notifications by email or slack
notify:
email:
recipients: [[email protected]]
slack:
users: []
channels: []
webhooks: []
```
4 changes: 3 additions & 1 deletion docs/docs/reference/project-files/explore-dashboards.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,9 @@ In your Rill project directory, create a explore dashboard, `<dashboard_name>.ya

**`metrics_view`** — Refers to the metrics view resource _(required)_.

**`title`** — Refers to the display name for the dashboard _(required)_.
**`title`** — Refers to the display name for the dashboard [deprecated, use `display_name`] _(required)_.

**`display_name`** - Refers to the display name for the metrics view _(required)_.

**`description`** - A description for the project. _(optional)_.

Expand Down
4 changes: 3 additions & 1 deletion docs/docs/reference/project-files/metrics-view.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,9 @@ In your Rill project directory, create a metrics view, `<metrics_view>.yaml`, fi

**`type`** — Refers to the resource type and must be `metrics_view` _(required)_.

**`title`** — Refers to the display name for the metrics view _(required)_.
**`title`** — Refers to the display name for the metrics view [deprecated, use `display_name`] _(required)_.

**`display_name`** - Refers to the display name for the metrics view _(required)_.

**`description`** - A description for the project. _(optional)_.

Expand Down
4 changes: 3 additions & 1 deletion docs/docs/reference/project-files/rill-yaml.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ The `rill.yaml` file contains metadata about your project.

## Properties

**`title`** — the name of your project which will be displayed in the upper left hand corner
**`title`** — the name of your project which will be displayed in the upper left hand corner [deprecated, use `display_name`] _(required)_.

**`display_name`** - Refers to the display name for the metrics view _(required)_.

**`compiler`** — the Rill project compiler version compatible with your project files (currently defaults to: `rillv1`)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ sidebar_label: "Cloud Storage: Incremental Partitioned Models"
sidebar_position: 12
---

Now that we understand what Incremental Models and Partitions are, let's apply to them to our ClickHouse project.
Now that we understand what [Incremental Models](/build/incremental-models/#what-is-an-incremental-model) and [Partitions]/(build/incremental-models/#what-are-partitions) are, let's apply to them to our ClickHouse project.

Since our ClickHouse data is hosted in GCS, we will be using glob based partitions, instead of the example's sql select statements.

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
---
title: "Increment Model based on a state from Data Warehouses"
description: "Getting Started with Partitions"
sidebar_label: "Data Warehouse: Increment Models"
sidebar_position: 12
---

Another advanced concept within Rill is using [Incremental Models](/build/incremental-models/#what-is-an-incremental-model) with a state defined.

:::tip requirements
You will need to setup the connection to your data warehouse, depending on the connection please refer to [our documentation](https://docs.rilldata.com/reference/connectors/).

In this example we use a DATE column as our defining state but depending on your data, you can use any defining column.

:::

## Understanding States in Models

Here’s how it works at a high level:

- **State Definition**: Based on a SQL query, defines a key that allows you to increment your model by
- **Execution Strategy**:
- **Full Refresh**: Runs without incremental processing.
- **Incremental Refresh**: Run incrementally based on the state defined, following the output connector's `incremental_strategy` (either append or merge for SQL connectors).

### Let's create a basic partitions model.

:::note Example
In this example, we are using a sample dataset that exists in Big Query: rilldata.ssb_100.date.
In this case our table is not getting updated, so instead we'll modify the SQL to show you how incremental works.
:::


1. Create a YAML file: `SQL_incremental_tutorial.yaml`

2. Use the following contents to create your own model.
```yaml
type: model
materialize: true

connector: "bigquery" #or "snowflake"

incremental: true
state:
sql: SELECT MAX(DATE) as max_date FROM SQL_incremental_tutorial #should be the name of the current model

sql: |
SELECT *,
PARSE_DATE('%Y%m%d', CAST(D_DATEKEY AS STRING)) AS DATE
FROM rilldata.ssb_100.date
{{if incremental}} # when incremental refreshing this part of the SQL is used.
WHERE PARSE_DATE('%Y%m%d', CAST(D_DATEKEY AS STRING)) = '{{.state.max_date}}' #normally would want to set this to where DATE > '{{.state.max_date}}' to only append new rows.
{{else}}
LIMIT 10 #restricts the full refresh to only 10 rows, so when we run incremental, its easy to tell the difference.
{{end}}
output:
connector: duckdb
incremental_strategy: append #merge, requires unique_key
#unique_key: [column_name] #if strategy is merge
```

3. In the UI, try refreshing both incrementally and fully to see the difference in the model that loads.
- when selecting a full refresh, only 10 rows should be returned.
- when selecting incremental refresh, it will **append** values to the inital 10 values in the full refresh.

![img](/img/tutorials/302/data-warehouse-refresh.png)

:::note Partition vs. State
Unlike partitions, states do not paritition the dataset per refresh so you will not be able to via the UI or CLI, see if there is a specific partition that errored and manually refresh this. In the cases of data disrecpancies in a state incremented model, please run a full refresh.
:::

import DocsRating from '@site/src/components/DocsRating';

---
<DocsRating />
Loading

0 comments on commit 5572d98

Please sign in to comment.