Skip to content
This repository has been archived by the owner on Aug 13, 2024. It is now read-only.

Commit

Permalink
Use SSC 0.12.0 and SE 0.13.0 in the Docker Compose example (closes #32)
Browse files Browse the repository at this point in the history
  • Loading branch information
BenFradet committed Jan 2, 2018
1 parent c13a5bf commit d60698c
Show file tree
Hide file tree
Showing 4 changed files with 19 additions and 15 deletions.
21 changes: 9 additions & 12 deletions example/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,25 +4,22 @@ This folder contains a Docker Compose example for the Snowplow realtime pipeline

## Introduction

This Docker Compose example bundles the following components in two distinct containers:
This Docker Compose example bundles the following Snowplow components in two distinct containers:

- [Scala Stream Collector][ssc]
- [Stream Enrich][se]

As is, the configuration files make the following assumptions regarding Kinesis streams:
Additionally, they make use of [NSQ][nsq] topics to store events, the NSQ components are running
in three different containers:

- The `snowplow-raw` stream exists and is used to store the collected events
- The `snowplow-enriched` stream exists and is used to store the enriched events
- The `snowplow-bad` stream exists and is used to store the events which failed validation

All those streams being located in `us-east-1`. To authenticate the components, the
[DefaultAWSCredentialsProviderChain][dacpc] is used.

Feel free to modify those configuration files to suit your needs.
- nsqd: the daemon in charge of receiving, queueing and delivering messages
- nsqlookupd: the daemon taking care of managing who produces and consumes what
- nsqadmin: a web UI to perform administrative tasks as well as giving an overview of the NSQ
topology

## Usage

Once you have configured the configuration files to your liking, you can launch those two components
Once you have modified the configuration files to your liking, you can launch those two components
with:

```bash
Expand Down Expand Up @@ -52,6 +49,6 @@ limitations under the License.
[ssc]: https://github.com/snowplow/snowplow-docker/tree/master/scala-stream-collector
[se]: https://github.com/snowplow/snowplow-docker/tree/master/stream-enrich

[dacpc]: http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/DefaultAWSCredentialsProviderChain.html
[nsq]: http://nsq.io/

[license]: http://www.apache.org/licenses/LICENSE-2.0
4 changes: 2 additions & 2 deletions example/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ services:
- "4171:4171"

scala-stream-collector:
image: snowplow-docker-registry.bintray.io/snowplow/scala-stream-collector:0.11.0
image: snowplow-docker-registry.bintray.io/snowplow/scala-stream-collector:0.12.0
command: [ "--config", "/snowplow/config/config.hocon" ]
depends_on:
- nsqd
Expand All @@ -51,7 +51,7 @@ services:
- "SP_JAVA_OPTS=-Xms512m -Xmx512m"

stream-enrich:
image: snowplow-docker-registry.bintray.io/snowplow/stream-enrich:0.12.0
image: snowplow-docker-registry.bintray.io/snowplow/stream-enrich:0.13.0
command: [
"--config", "/snowplow/config/config.hocon",
"--resolver", "file:/snowplow/config/resolver.json",
Expand Down
8 changes: 7 additions & 1 deletion example/scala-stream-collector-config/config.hocon
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,14 @@ collector {

cookieBounce {
enabled = false
name = "n3pc"
name = n3pc
fallbackNetworkUserId = "00000000-0000-4000-A000-000000000000"
forwardedProtocolHeader = "X-Forwarded-Proto"
}

redirectMacro {
enabled = false
placeholder = "[TOKEN]"
}

streams {
Expand Down
1 change: 1 addition & 0 deletions example/stream-enrich-config/config.hocon
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ enrich {
rawChannel = enrich
host = nsqd
port = 4150
lookupHost = nsqlookupd
lookupPort = 4161
}

Expand Down

0 comments on commit d60698c

Please sign in to comment.