Merge with integration branch

alertlogic · Jan 25, 2018 · 76adb14 · 76adb14
2 parents d8c9db7 + b34fb00
commit 76adb14
Show file tree

Hide file tree

Showing 5 changed files with 362 additions and 41 deletions.
diff --git a/README.md b/README.md
@@ -51,6 +51,24 @@ lambda function into a zip file.
 
 Please use the following [code style](https://github.com/airbnb/javascript) as much as possible.
 
+# Debugging
+
+To get a debug trace, set an Node.js environment variable called DEBUG and
+specify the JavaScript module/s to debug.
+
+E.g.
+
+```
+export DEBUG=*
+export DEBUG=index
+```
+
+Or set an environment variable called "DEBUG" in your AWS stack (using the AWS 
+console) for the "alertlogic-cwe-collector" AWS Lambda function, with 
+value "index" or "*".
+
+See [debug](https://www.npmjs.com/package/debug) for further details.
+
 # Known Issues/ Open Questions
 
 - TBD.

diff --git a/cfn/README-GD.md b/cfn/README-GD.md
@@ -1,6 +1,6 @@
 # guardduty.template
 
-Alert Logic Amazon Web Services (AWS) CloudWatch Events (CWE) Collector CloudFormation templates.
+Alert Logic Amazon Web Services (AWS) CloudWatch Events (CWE) Collector CloudFormation template.
 
 # Overview
 
@@ -12,6 +12,12 @@ AWS usage statistics techniques to identify unexpected and potentially unauthori
 Such activity includes escalations of privileges, uses of exposed credentials, or communication with 
 malicious IPs, URLs, or domains. GuardDuty informs you of the status of your AWS infrastructure and applications by producing security `findings`. 
 
+# Before you begin
+
+This procedure requires administrative permissions in AWS and your Alert Logic Cloud Insight account.  You also need to download the Alert Logic custom CFT to your local machine from [the Alert Logic public github repository](https://github.com/alertlogic/cwe-collector/blob/master/cfn/guardduty.template).
+
+Windows systems also require PowerShell version 3.0 or later. If you have an earlier version of PowerShell, we suggest you [upgrade it](https://docs.microsoft.com/en-us/powershell/scripting/setup/installing-windows-powershell#upgrading-existing-windows-powershell) to version 3.0 or later. 
+
 # Installation
 
 To install the GuardDuty events collector:
@@ -29,18 +35,24 @@ To enable Amazon GuardDuty events, see [Setting Up Amazon GuardDuty](https://doc
 
 ## Alert Logic Access key creation
 
-**Before you begin:** Be sure you have an Alert Logic Cloud Insight account with administrator permissions. Log into the Cloud Insight console as an administrator [here](https://console.cloudinsight.alertlogic.com/#/login).
-In order to verify the user has administrator permissions:
+### Verify permissions
+
+Log into the Cloud Insight console as an administrator [here](https://console.cloudinsight.alertlogic.com/#/login) to verify administrator permissions:
+
+1. In the Cloud Insight console, click the user name at the top-right corner.
+1. In the drop-down menu, click **Users**.
+1. Select the user in `AIMS User` section. **Note:** you can start typing a name in the search box to find the appropriate user.
+1. Verify the `user role` as listed under `Edit an AIMS User` has the `Administrator` role selected.
 
-1. Once logged in click on the user name at the top-right corner of Cloud Insight console.
-1. In the drop-down menu, click `Users`.
-1. Select the user in `AIMS User` section. **Note** you can start typing a name in the search box to find an appropriate user.
-1. Once found check that `user role` list under the `Edit an AIMS User` section has `Administrator` role selected.
+### Key creation
 
-The following procedure assumes a Linux-based local machine using [curl](https://curl.haxx.se/) and 
-[jq](https://stedolan.github.io/jq/). For Windows please use command line and windows versions of [curl](https://curl.haxx.se/download.html) and [jq](https://stedolan.github.io/jq/download/).
+Use the instructions below that match your operating system: Unix (MacOS, Linux) or Windows.
 
-From the bash command line, type the following commands, where `<email address>` is your Alert Logic Cloud Insight email address you use to log in, and then enter your password when prompted:
+#### Unix (MacOS, Linux)
+
+The following procedure assumes a Unix-based local machine using [curl](https://curl.haxx.se/) and [jq](https://stedolan.github.io/jq/).
+
+From the bash command line, type the following commands, where `<email address>` is the Alert Logic Cloud Insight email address you use to log in. Enter your password when prompted.
 
 ```
 export AL_USERNAME='<email address>'
@@ -55,68 +67,106 @@ An example of a successful response is:
 }
 ```
 
-**Important:** If the command returns no output, verify your Alert Logic account has administrator permissions. Click [here](https://console.cloudinsight.alertlogic.com/api/aims/) for more information about AIMS APIs.
+Make a note of the `access_key_id` and `secret_key` values, which you need to deploy the CloudFormation template to your AWS account. 
+
+**Necessary role error**
 
-Make a note of the `access_key_id` and `secret_key` values, which you need when you deploy the CloudFormation template to your AWS account. 
+If the command returns an error about not having the necessary role, please verify your Alert Logic account has administrator permissions. Click [here](https://console.cloudinsight.alertlogic.com/api/aims/) for more information about AIMS APIs.
 
-**Note:** Each user can create only five access keys. If a "limit exceeded" response appears, you must delete one or more access keys before you can create new keys. 
+**"Limit exceeded" error**
+
+Each user can create only five access keys. If a "limit exceeded" response appears, you must delete one or more access keys before you can create new keys. 
+
+1. Type the following command to list access keys:
+    ```
+    curl -s -X GET -H "x-aims-auth-token: $AL_TOKEN" https://api.global-services.global.alertlogic.com/aims/v1/$AL_ACCOUNT_ID/users/$AL_USER_ID/access_keys | jq
+    ```
+1. Use the selected access_key_id in the following command to delete the key:
+    ```
+    curl -X DELETE -H "x-aims-auth-token: $AL_TOKEN" https://api.global-services.global.alertlogic.com/aims/v1/$AL_ACCOUNT_ID/users/$AL_USER_ID/access_keys/<ACCESS_KEY_ID_HERE>
+    ```
+
+
+#### Windows
+
+**Note:** These instructions require PowerShell 3.0 or later.
+
+In the PowerShell console, please type the following commands. Enter your Alert Logic Cloud Insight email address and password when prompted.
 
-**Type the following command to list access keys:**
 ```
-curl -s -X GET -H "x-aims-auth-token: $AL_TOKEN" https://api.global-services.global.alertlogic.com/aims/v1/$AL_ACCOUNT_ID/users/$AL_USER_ID/access_keys | jq
+[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; $creds = Get-Credential -Message "Please enter your Alert Logic Cloud Insight email address and password"; $unsecureCreds = $creds.GetNetworkCredential(); $base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $unsecureCreds.UserName,$unsecureCreds.Password))); Remove-Variable unsecureCreds; $AUTH = Invoke-RestMethod -Method Post -Headers @{"Authorization"=("Basic {0}" -f $base64AuthInfo)} -Uri https://api.global-services.global.alertlogic.com/aims/v1/authenticate ; Remove-Variable base64AuthInfo; $AL_ACCOUNT_ID = $AUTH.authentication.account.id; $AL_USER_ID = $AUTH.authentication.user.id; $AL_TOKEN = $AUTH.authentication.token; if (!$AL_TOKEN) { Write-Host "Authentication failure"} else { $ROLES_RESP = Invoke-RestMethod -Method Get -Headers @{"x-aims-auth-token"=$AL_TOKEN} -Uri https://api.global-services.global.alertlogic.com/aims/v1/$AL_ACCOUNT_ID/users/$AL_USER_ID/roles ; $ROLES = $ROLES_RESP.roles.name; if ($ROLES -ne "Administrator" ) { Write-Host "Your user doesn’t have Administrator role. Assigned role is '$ROLES'" } else { $ACCESS_KEY = Invoke-RestMethod -Method Post -Headers @{"x-aims-auth-token"=$AL_TOKEN} -Uri https://api.global-services.global.alertlogic.com/aims/v1/$AL_ACCOUNT_ID/users/$AL_USER_ID/access_keys ; Write-Host $ACCESS_KEY } }
 ```
 
-**Use the selected access_key_id in the following curl command to delete the key:**
+An example of a successful response is:
 
 ```
-curl -X DELETE -H "x-aims-auth-token: $AL_TOKEN" https://api.global-services.global.alertlogic.com/aims/v1/$AL_ACCOUNT_ID/users/$AL_USER_ID/access_keys/<ACCESS_KEY_ID_HERE>
+@{access_key_id=712c0b413eef41f6; secret_key=1234567890b3eea8880d292fb31aa96902242a076d3d0e320cc036eb51bf25ad}
 ```
 
+Make a note of the `access_key_id` and `secret_key` values, which you need to deploy the CloudFormation template to your AWS account.
+
+**Necessary role error**
+
+If the command returns an error about not having the necessary role, please verify your Alert Logic account has administrator permissions. Click [here](https://console.cloudinsight.alertlogic.com/api/aims/) for more information about AIMS APIs.
+
+**"Limit exceeded" error**
+
+Each user can create only five access keys. If a "limit exceeded" response appears, you must delete one or more access keys before you can create new keys.
+
+1. Type the following command to list access keys:
+    ```
+    Invoke-RestMethod -Method Get -Headers @{"x-aims-auth-token"=$AL_TOKEN} -Uri https://api.global-services.global.alertlogic.com/aims/v1/$AL_ACCOUNT_ID/users/$AL_USER_ID/access_keys
+    ```
+1. Use the selected `access_key_id` in the following command to delete the key:
+    ```
+    Invoke-RestMethod -Method Delete -Headers @{"x-aims-auth-token"=$AL_TOKEN} -Uri https://api.global-services.global.alertlogic.com/aims/v1/$AL_ACCOUNT_ID/users/$AL_USER_ID/access_keys/<ACCESS_KEY_ID_HERE>
+    ```
+
+
 ## CloudFormation template deployment
 
 The Alert Logic CWE collector deploys to a single AWS region. To collect from 
-multiple AWS regions, you must either install the collector in each target region, or 
+multiple AWS regions, you must either install the collector in each target region or 
 set up GuardDuty collection across regions. For more information, see: [Setting up GuardDuty across
 regions and accounts](https://docs.aws.amazon.com/guardduty/latest/ug/guardduty_accounts.html).
 
-**Note:** This procedure assumes setup in the AWS `us-east-1` region 
-using the Alert Logic Cloud Insight [US console](https://console.cloudinsight.alertlogic.com/#/login). If
-your setup is in a European region (e.g. `eu-east-1`), use the 
-[UK console](https://console.cloudinsight.alertlogic.co.uk/#/login).
+### Use the Amazon console to deploy
+
+**Note:** This procedure assumes setup in the AWS `us-east-1` region using the Alert Logic Cloud Insight [US console](https://console.cloudinsight.alertlogic.com/#/login). If your setup is in a European region (e.g., `eu-east-1`), use the [UK console](https://console.cloudinsight.alertlogic.co.uk/#/login).
 
 1. Log in to the [AWS Management Console](https://aws.amazon.com/console/) with an AWS account that has AWS administrator privileges. 
 1. Select the region in which you want to deploy the CFT.
-1. Click `Services`->`CloudFormation`->`Create Stack`.
-1. In the `Choose a template` section select `Specify an Amazon S3 template URL`, use the following URL and click `Next`:
+1. Click **Services**->**CloudFormation**->**Create Stack**.
+1. In the `Choose a template` section select **Specify an Amazon S3 template URL** and enter the following URL.
 `https://s3.amazonaws.com/alertlogic-collectors-us-east-1/cfn/guardduty.template`
+1. Click **Next**.
 1. On the `Specify Details` window, provide the following required parameters:
    - `Stack name` - Any name you have used for creating an AWS stack
    - `AccessKeyId` - `access_key_id` returned from AIMs [above](#alert-logic-access-key-creation)
    - `AlApiEndpoint` - usually `api.global-services.global.alertlogic.com` 
    - `AlDataResidency` - usually `default`
    - `SecretKey` - `secret_key` returned from AIMs [above](#alert-logic-access-key-creation)
-1. Click Next. 
-1. On the Options panel, click Next.
-1. In the Review panel, perform a predeployment check. 
-1. Select "I acknowledge that AWS CloudFormation might create IAM resources," and then click Create.
-1. On the CloudFormation, Stacks panel, filter based on the stack name you created, and then 
+1. Click **Next**. 
+1. On the `Options` panel, click **Next**.
+1. In the `Review` panel, perform a predeployment check. 
+1. Select **"I acknowledge that AWS CloudFormation might create IAM resources"**, then click **Create**.
+1. On the CloudFormation, `Stacks` panel, filter based on the stack name you created, and then 
 select your stack by name.
 
+If the deployment was successful, the status appears as: CREATE_COMPLETE. If the deployment was not successful, 
+see [Troubleshooting Installation Issues](#troubleshooting-installation-issues) below.
+
 **Note:** Only one collector installation is allowed per AWS region. If you try to deploy the template multiple times in the same region, if will fail with the following error:
 
 Status | Type | Logical ID | Status Reason
 --|--|--|--
 CREATE_FAILED | AWS::Lambda::Function | CollectLambdaFunction | alertlogic-cwe-collector already exists in stack arn:aws:cloudformation:us-east-1:123456789101:stack/test-one/f9536300-d12b-11e7-ac98-50d5cd16c68e
 
 
-If the deployment was successful, the status appears as: CREATE_COMPLETE. If the deployment was not successful, 
-see [Troubleshooting Installation Issues](#troubleshooting-installation-issues) below.
-
-#### Use a Command Line to deploy
+### Use a Command Line to deploy
 
 Follow these steps to deploy the Alert Logic custom template using the [AWS CLI](https://aws.amazon.com/cli/).
 
-1. Download the Alert Logic custom CFT to your local machine from [the Alert Logic public github repository](https://github.com/alertlogic/cwe-collector/blob/master/cfn/guardduty.template). 
 1. In the command line, type the following command, where the required parameters are:
     - `stack-name` - Any name you have used to create an AWS stack
     - `AccessKeyId` - `access_key_id` returned from AIMs [above](#alert-logic-access-key-creation)
@@ -129,14 +179,13 @@ Follow these steps to deploy the Alert Logic custom template using the [AWS CLI]
 
 ## Deployment verification
 
-1. Log into the Alert Logic Cloud Insight console.
-**Note:** You must log in with an account that has administrator permissions.
+1. Log into the Alert Logic Cloud Insight console using an account that has administrator permissions.
     - Use the [US console](https://console.cloudinsight.alertlogic.com/#/login) for regions in the US and associated geographical regions.
     - Use the [UK console](https://console.cloudinsight.alertlogic.co.uk/#/login) for regions in Europe and other regions not in the US.
 1. If you have not already created a Cloud Insight deployment, follow the instructions [here](https://docs.alertlogic.com/gsg/amazon-web-services-cloud-insight-get-started.htm) to do so for the AWS account and region where you installed the CFT.
-1. Verify successful deployment by checking the Incident list in Alert Logic Cloud Insight UI. The list should be populated with Incidents that correspond to recent Amazon GuardFindings which are displayed in Amazon GuardDuty console.
+1. Verify successful deployment by checking the Incident list in the Alert Logic Cloud Insight UI. The list should be populated with Incidents that correspond to recent Amazon GuardFindings, which are displayed in the Amazon GuardDuty console.
 
-## Troubleshooting Installation Issues
+## Troubleshooting installation issues
 
 If installation through the [AWS Management Console](https://aws.amazon.com/console/) is not successful, you can access 
 `CloudFormation`->`Stacks`->`Stack Detail` (by selecting your stack name from the list) to see detailed
@@ -149,8 +198,8 @@ If installation through the AWS CLI is not successful, issue the following comma
 aws cloudformation describe-stack-events --stack-name my-new-stack
 ```
 
-1. If `GetEndpointsLambdaFunction` fails, an issue could exist with the AIMs access key id or the secret key
- you provided. Be sure the key id is correct, your secret key is valid, and your user account has administrative permissions for the Alert Logic Cloud Insight account.
+1. If `GetEndpointsLambdaFunction` fails, an issue could exist with the `access_key_id` or the `secret_key`
+ you provided. Be sure the `access_key_id` is correct, your `secret_key` is valid, and your user account has administrative permissions for the Alert Logic Cloud Insight account.
 1. Other issues, TBD.
 
 

diff --git a/index.js b/index.js
@@ -9,6 +9,7 @@
  * -----------------------------------------------------------------------------
  */
 
+const debug = require('debug') ('index'); 
 const https = require('https');
 const util = require('util');
 const AWS = require('aws-sdk');
@@ -73,6 +74,13 @@ function getKinesisData(event, callback) {
 function filterGDEvents(cwEvents, callback) {
     async.filter(cwEvents,
         function(cwEvent, filterCallback){
+            if (cwEvent.source && cwEvent.source === 'aws.guardduty') {
+                debug(`DEBUG0002: filterGDEvents - including event: ` +
+                    `${JSON.stringify(cwEvent)} `);
+            } else {
+                debug(`DEBUG0003: filterGDEvents - filtering out event: ` +
+                    `${JSON.stringify(cwEvent)} `); 
+            };
             return filterCallback(null, cwEvent.source && 
                 cwEvent.source === 'aws.guardduty');
         },
@@ -305,6 +313,7 @@ function getStatistics(context, event, finalCallback) {
 
 
 exports.handler = function(event, context) {
+    debug("DEBUG0001: Received event: ", JSON.stringify(event));
     switch (event.RequestType) {
         case 'ScheduledEvent':
             return processScheduledEvent(event, context);