Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Audience observability and monitoring features #7312

Open
wants to merge 6 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 11 additions & 7 deletions src/connections/delivery-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,6 @@ title: Delivery Overview

Delivery Overview is a visual observability tool designed to help Segment users diagnose event delivery issues for any cloud-streaming destination receiving events from cloud-streaming sources.

> info "Delivery Overview for RETL destinations and Engage Audience Syncs currently in development"
> This means that Segment is actively developing Delivery Overview features for RETL destinations and Engage Audience syncs. Some functionality may change before Delivery Overview for these integrations becomes generally available.
>
> Delivery Overview is generally available for streaming connections (cloud-streaming sources and cloud-streaming destinations) and in public beta for storage destinations. Some metrics specific to storage destinations, like selective syncs, failed row counts, and total rows seen, are not yet available.
> All users of Delivery Overview have access to the Event Delivery tab, and can configure delivery alerts for their destinations.


## Key features

Delivery Overview has three core features:
Expand Down Expand Up @@ -63,6 +56,17 @@ The following image shows a storage destination with 23 partially successful syn

![A screenshot of the Delivery Overview tab for a Storage destination, with the Failed to sync step selected and a table of partially successful syncs.](images/delivery-overview-storage-destinations.png)

#### Destinations connected to Audiences
Destinations connected to an Audience have the following steps in the pipeline view:
- **Events from audience**<sup>*</sup>: Events that Segment created for your activation. The number of events for each compute depends on the changes detected in your audience membership.
- **Filtered at source**: Events discarded by Protocols: either by the [schema settings](/docs/protocols/enforce/schema-configuration/) or [Tracking Plans](/docs/protocols/tracking-plan/create/).
- **Filtered at destination**: If any events aren’t eligible to be sent (for example, due to destination filters, insert function logic, and so on), Segment displays them at this step.
- **Events pending retry**: A step that reveals the number of events that are awaiting retry. Unlike the other steps, you cannot click into this step to view the breakdown table.
- **Failed delivery**: Events that Segment _attempted_ to deliver to your destination, but that ultimately _failed_ to be delivered. Failed delivery might indicate an issue with the destination, like invalid credentials, rate limits, or other error statuses received during delivery.
- **Successful delivery**: Events that Segment successfully delivered to your destination. You’ll see these events in your downstream integrations.

<sup>*</sup>_The "Events from audience" step is currently only available for Linked Audiences._

### Breakdown table
The breakdown table provides you with greater detail about the selected events.

Expand Down
72 changes: 72 additions & 0 deletions src/engage/audiences/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,78 @@ Engage then processes your realtime Audience or Trait edits. While the edit task
> warning ""
> You can't edit an audience to include anonymous users. If you need to include anonymous profiles, recreate the audience with the appropriate conditions

## Monitor the health of your Audience syncs

Use Segment's [Delivery Overview](#delivery-overview) and [Alerting](#alerting) features to monitor the health of your Audience syncs and get notifications when event volume spikes or drops.

### Delivery Overview

Delivery Overview is a visual observability tool designed to help Segment users diagnose event delivery issues for any event-streaming destination receiving events from Engage Audiences.

Delivery Overview has three core features:
- [Pipeline view](/docs/connections/delivery-overview/#pipeline-view): A visual overview of each step your data takes during the delivery process - from when your source receives audience events to when events are successfully delivered to your connected destination.
- [Breakdown table](/docs/connections/delivery-overview/#breakdown-table): If you select a step in the pipeline view, you can see more detail about the events that were processed at each pipeline step.
- [Discard table](/docs/connections/delivery-overview/#discard-table): If you select an event in a breakdown table, you can see more details about the events that failed or were filtered out of your process and allows you to inspect samples of them.

For more information about the breakdown and discard tables, see the [Delivery Overview](/docs/connections/delivery-overview/) documentation.

To view Delivery Overview for an Audience:
1. From your Segment workspace's home page, navigate to **Engage > Audiences**.
2. Find an Audience, click the **(...)** menu, and select Delivery Overview.
3. On the Delivery Overview page, select the Audience dropdown to filter by a specific Audience, select the Date range dropdown to filter by a specific time period, or toggle the Show metrics as percentages toggle on to view your metrics as percentages.

#### Steps in the pipeline view

By default, Segment displays Delivery Overview information for all Audiences connected to your destination. You can filter your Delivery Overview pipeline view by an individual Audience for more granular data.

You can also further refine the data displayed on the pipeline view using the time picker and the metric toggle, located under the destination header. With the time picker, you can specify a time period (last 10 minutes, 1 hour, 24 hours, 7 days, 2 weeks, or a custom date range over the last two weeks) for which you’d like to see data. With the metric toggle, you can switch between seeing metrics represented as percentages (for example, _85% of events_ or _an 133% increase in events_) or as counts (_13 events_ or _an increase of 145 events_.) Delivery Overview shows percentages by default.

> info "Linked Audiences have additional filtering functionality"
> Linked Audiences users can filter the Delivery Overview event pipeline by [Linked Audience events](/docs/engage/audiences/linked-audiences/#step-2c-define-how-and-when-to-trigger-an-event-to-your-destination). For more information, see the [Linked Audiences](/docs/engage/audiences/linked-audiences/#delivery-overview-for-linked-audiences) documentation.

Audiences have the following steps in the pipeline view:
- **Events from audience**<sup>*</sup>: Events that Segment created for your activation. The number of events for each compute depends on the changes detected in your audience membership.
- **Filtered at source**: Events discarded by Protocols: either by the [schema settings](/docs/protocols/enforce/schema-configuration/) or [Tracking Plans](/docs/protocols/tracking-plan/create/).
- **Filtered at destination**: If any events aren’t eligible to be sent (for example, due to destination filters, insert function logic, and so on), Segment displays them at this step.
- **Events pending retry**: A step that reveals the number of events that are awaiting retry. Unlike the other steps, you cannot click into this step to view the breakdown table.
- **Failed delivery**: Events that Segment _attempted_ to deliver to your destination, but that ultimately _failed_ to be delivered. Failed delivery might indicate an issue with the destination, like invalid credentials, rate limits, or other error statuses received during delivery.
- **Successful delivery**: Events that Segment successfully delivered to your destination. You’ll see these events in your downstream integrations.

<sup>*</sup>_The "Events from audience" step is currently only available for Linked Audiences._

### Alerting

Create alerts related to the performance and throughput of Audience syncs and receive in-app, email, and Slack notifications when event volume fluctuations occur.

> info "Generate a Slack webhook to receive Slack notifications"
> To receive an alert in a Slack channel, you must first create a Slack webhook. For more information about Slack webhooks, see Slack's [Sending messages using incoming webhooks](https://api.slack.com/messaging/webhooks){:target="_blank”} documentation.

To access Audience alerting, navigate to **Engage > Audiences**, select an Audience, and click the Alerts tab.

On the Alerts tab, you can create new alerts and view all active alerts for this connection. You can only edit or delete the alerts that you create, unless you have the [Workspace Owner role](/docs/segment-app/iam/roles/).

#### Activation event health spikes or drops

You can create an Activation event health spikes or drops alert that notifies you when events sent from your audience to a downstream destination have failures to a destination above a certain threshold. For example, if you set a change percentage of 4% and your destination received 100 events from your Audience over the first 24 hours, Segment would notify you the following day if your destination ingested fewer than 96 or more than 104 events.

To create an Activation event health spikes or drops alert:
1. From your Segment workspace's home page, navigate to **Engage > Audiences**.
2. Select the Audience you want to create an alert for, select the Alerts tab, and click **Create alert**.
3. On the Create alert sidesheet, select the destination for which you'd like to monitor event health.
4. Enter a percentage of activation event health that you'd like to be notified for.
5. Select one or more of the following alert channels:
- **Email**: Select this to receive notifications at the provided email address.
- **Slack**: Select this to send alerts to one or more channels in your workspace.
- **In-app**: Select this to receive notifications in the Segment app. To view your notifications, select the bell next to your user icon in the Segment app.
6. Click **Save**.

To make changes to a Activation event health spikes or drops alert, select the icon in the Actions column for the alert and click **Edit**.

To delete a Activation event health spikes or drops alert, select the icon in the Actions column for the alert and click **Delete**.

> info "Deleting alerts created by other users requires Workspace Owner role"
> All users can delete alerts that they created, but only those with [Workspace Owner role](/docs/segment-app/iam/roles/) can delete alerts created by other users.

## Access your Audiences using the Profiles API

You can access your Audiences using the Profile API by querying the `/traits` endpoint. For example, you can query for `high_value_user` property with the following `GET` request:
Expand Down
20 changes: 14 additions & 6 deletions src/engage/audiences/linked-audiences.md
Original file line number Diff line number Diff line change
Expand Up @@ -241,12 +241,20 @@ With your Linked Audience activated, follow these steps to monitor your activati

### Delivery Overview for Linked Audiences

Delivery Overview shows you four steps in your data activation pipeline:

- **Events from Audience**: Events that Segment created for your activation. The number of events for each compute depends on the changes detected in your audience membership.
- **Filtered at Destination**: The activation pipeline is rich with features that let you control which events make it to the destination. If any events aren't eligible to be sent (for example, due to destination filters, insert function logic, and so on), Segment will show them in Filtered at Destination.
- **Failed Delivery**: Events that Segment attempted but failed to deliver to your destination. Failed Delivery indicates an issue with the destination, like invalid credentials, rate limits, or other error statuses received during delivery.
- **Successful Delivery**: Events that Segment successfully delivered to your destination. You'll see these events in your downstream integration.
In addition to the standard Audience observability provided by [Delivery Overview](/docs/engage/audiences/#delivery-overview), Linked Audiences can filter Delivery Overview's pipeline view by [Linked Audience events](/docs/engage/audiences/linked-audiences/#step-2c-define-how-and-when-to-trigger-an-event-to-your-destination).

To filter by events:
1. From your Segment workspace's home page, navigate to **Engage > Audiences**.
2. Find an Audience, click the **(...)** menu, and select Delivery Overview.
3. On the Delivery Overview page, select the Linked audience event dropdown to filter by a specific event.

Linked Audiences have the following steps in Delivery Overview's pipeline view:
- **Events from audience**: Events that Segment created for your activation. The number of events for each compute depends on the changes detected in your audience membership.
- **Filtered at source**: Events discarded by Protocols: either by the [schema settings](/docs/protocols/enforce/schema-configuration/) or [Tracking Plans](/docs/protocols/tracking-plan/create/).
- **Filtered at destination**: If any events aren’t eligible to be sent (for example, due to destination filters, insert function logic, and so on), Segment displays them at this step.
- **Events pending retry**: A step that reveals the number of events that are awaiting retry. Unlike the other steps, you cannot click into this step to view the breakdown table.
- **Failed delivery**: Events that Segment _attempted_ to deliver to your destination, but that ultimately _failed_ to be delivered. Failed delivery might indicate an issue with the destination, like invalid credentials, rate limits, or other error statuses received during delivery.
- **Successful delivery**: Events that Segment successfully delivered to your destination. You’ll see these events in your downstream integrations.

## Maintaining Linked Audiences

Expand Down