Skip to content

alertlogic/paws-collector

Folders and files

NameName
Last commit message
Last commit date

Latest commit

31d33f8 · Dec 5, 2019

History

24 Commits
Nov 27, 2019
Dec 4, 2019
Nov 26, 2019
Nov 27, 2019
Nov 26, 2019
Nov 26, 2019
Nov 27, 2019
Nov 26, 2019
Nov 27, 2019
Nov 14, 2019
Nov 27, 2019
Nov 26, 2019
Dec 5, 2019
Dec 4, 2019

Repository files navigation

paws-collector

Alert Logic AWS Based API Poll (PAWS) Log Collector Library.

Overview

This repository contains the AWS JavaScript Lambda function and CloudFormation Template (CFT) for deploying a log collector in AWS which will poll 3rd party service API to collect and forward logs to the Alert Logic CloudInsight backend services.

Installation

Refer to CF template readme for installation instructions.

How it works

Update Trigger

The Updater is a timer triggered function that runs a deployment sync operation every 12 hours in order to keep the collector lambda function up to date. The Updater syncs from the Alert Logic S3 bucket where you originally deployed from.

Collection Trigger

The Collector function is an AWS lambda function which is triggered by SQS which contains collection state message. During each invocation the function polls 3rd party service log API and sends retrieved data to AlertLogic Ingest service for further processing.

Checkin Trigger

The Checkin Scheduled Event trigger is used to report the health and status of the Alert Logic AWS lambda collector to the Azcollect back-end service based on an AWS Scheduled Event that occurs every 15 minutes.

Development

Build

Clone this repository and build a lambda package by executing:

$ git clone https://github.com/alertlogic/paws-collector.git
$ cd paws-collector
$ make deps test package

Build collector for Okta

Clone this repository and build a lambda package by executing:

$ git clone https://github.com/alertlogic/paws-collector.git
$ cd paws-collector/collectors/okta
$ make deps test package

The package name is al-okta-collector.zip

Debugging

To get a debug trace, set an Node.js environment variable called DEBUG and specify the JavaScript module/s to debug.

E.g.

export DEBUG=*
export DEBUG=index

Or set an environment variable called "DEBUG" in your AWS stack (using the AWS console) for a collector AWS Lambda function, with value "index" or "*".

See debug for further details.

Invoking locally

In order to invoke lambda locally please follow the instructions to install AWS SAM. AWS SAM uses default credentials profile from ~/.aws/credentials.

  1. Encrypt the key using aws cli:
aws kms encrypt --key-id KMS_KEY_ID --plaintext AIMS_SECRET_KEY
  1. Include the encrypted token, and KmsKeyArn that you used in Step 1 inside my SAM yaml:
    KmsKeyArn: arn:aws:kms:us-east-1:xxx:key/yyy
    Environment:
        Variables:
  1. Fill in environment variables in env.json (including encrypted AIMS secret key) and invoke locally:
cp ./local/env.json.tmpl ./local/env.json
vi ./local/env.json
make test
make sam-local
  1. Please see local/event.json for the event payload used for local invocation.