Enable deployment by terraform root module

.github/workflows/test.yml

@@ -0,0 +1,46 @@
+name: Terraform Test
+
+on:
+  pull_request:
+    branches:
+      - main
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    name: Integration test
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: terraform validate
+        uses: dflook/terraform-validate@v1
+
+      - name: terraform fmt
+        uses: dflook/terraform-fmt-check@v1
+
+      - name: terraform test
+        uses: dflook/terraform-test@v1
+        env:
+          CF_USER: ${{ secrets.CF_USER }}
+          CF_PASSWORD: ${{ secrets.CF_PASSWORD }}
+          TF_VAR_cf_user: ${{ secrets.CF_USER }}
+          TERRAFORM_PRE_RUN: |
+            apt-get update
+            apt-get install -y zip
+
+
+  lint-sandbox:
+    runs-on: ubuntu-latest
+    name: Lint sandbox
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: terraform validate
+        uses: dflook/terraform-validate@v1
+        with:
+          path: sandbox-deploy
+
+      - name: terraform fmt
+        uses: dflook/terraform-fmt-check@v1
+        with:
+          path: sandbox-deploy

README.md

@@ -86,7 +86,7 @@ in [Runner Execution Flow](https://gitlab.com/gitlab-org/gitlab-runner/-/tree/ma
   for details.
 
 * **A cloud.gov space to host the deployment service account** - Runners should only be deployed
-  into their own dedicated spaces. The terraform in terraform/runner-manager takes care of this.
+  into their own dedicated spaces. The terraform deployment takes care of this.
 
 * **Network access from your runner space to required Internet destinations** -
   Runners need to reach your GitLab server via HTTPS. They also need to be able
@@ -95,7 +95,7 @@ in [Runner Execution Flow](https://gitlab.com/gitlab-org/gitlab-runner/-/tree/ma
 
 ## Deploying and Troubleshooting
 
-See [the terraform README](terraform/runner-manager/README.md) for deployment instructions and troubleshooting tips.
+See [the terraform README](README.terraform.md) for deployment instructions and troubleshooting tips.
 
 ## TODO
 

terraform/runner-manager/README.md → README.terraform.md

@@ -5,9 +5,31 @@ Terraform for running GitLab CI/CD jobs on cloud.gov or another CloudFoundry bas
 * [Deploying](#deploying)
 * [Troubleshooting](#troubleshooting)
 * [Design Decisions](#design-decisions)
+* [Testing](#testing)
 
 ## Deploying
 
+Include this repository as a module in your terraform setup to deploy a gitlab-runner-cloudgov system.
+
+```terraform
+module "gitlab-runner" {
+    source = "github.com/gsa-tts/gitlab-runner-cloudgov?ref=main" # TODO: releases via tags
+
+    cf_space_prefix  = "SPACEPREFIX"
+    ci_server_token  = "TOKEN"
+    docker_hub_user  = "docker-username"
+    docker_hub_token = "docker-personal-access-token"
+    worker_egress_allowlist = [
+        "hostname.to.allow", # explanation of why it's allowed
+        "other.hostname.to.allow" # explanation of why it's allowed
+    ]
+}
+```
+
+### Local terraform use / sandbox deploy
+
+For local development, there is a `sandbox-deploy` module set up to deploy the root module that will use local state storage.
+
 1. Log in to cloud.gov and select your ORGNAME when prompted
     ```
     cf login -a api.fr.cloud.gov --sso
@@ -18,9 +40,14 @@ Terraform for running GitLab CI/CD jobs on cloud.gov or another CloudFoundry bas
     cf create-space SPACEPREFIX-mgmt
     ```
 
+1. Switch working directory
+    ```
+    cd sandbox-deploy
+    ```
+
 1. Create a [cloud.gov service account](https://cloud.gov/docs/services/cloud-gov-service-account/) with the `OrgManager` permission
     ```
-    ../create_service_account -s SPACEPREFIX-mgmt -u glr-local-deploy -m > secrets.auto.tfvars
+    ./create_service_account -s SPACEPREFIX-mgmt -u glr-local-deploy -m > secrets.auto.tfvars
     ```
 
 1. Copy `vars.tfvars-example` to `vars.auto.tfvars`.
@@ -52,23 +79,22 @@ for much more on GitLab CI/CD and runners.
 
 ### Viewing manager instance logs
 
-Problems with runner registration often requiring viewing it's logs.
+Problems with runner registration often requiring viewing its logs.
 
 ~~~
-cf logs --recent RUNNER-NAME
+cf target -s SPACEPREFIX-manager
+cf logs --recent devtools-runner-manager
 ~~~
 
 ### Dependency installs are not working, dependencies cannot be downloaded.
 
 The manager and workers run in [restricted-egress](https://cloud.gov/docs/management/space-egress/) spaces. There are two places to edit in order to allow traffic.
 
 1. If the runner-manager cannot download something, or the runner-workers are failing during the `prepare.sh` steps then the `local.devtools_egress_allowlist` in `main.tf` should be updated
-1. If the runner-workers cannot download a dependency required because of the programming language in use by the project, then it should likely be added to the `var.worker_egress_allowlist` in `vars.auto.tfvars`
+1. If the runner-workers cannot download a dependency required because of the programming language in use by the project, then it should likely be added to the `var.worker_egress_allowlist` in `vars.auto.tfvars` or your module include.
 
 It is also possible that additional configuration is required for the package manager in question to direct traffic over the proxy.
 
-## TODO
-
 ## Design Decisions
 
 ### Use environment variables to register gitlab-runner
@@ -77,3 +103,11 @@ Recent versions of `gitlab-runner` expose almost all initial configuration
 variables for the `register` subcommand as environment variables. This allows
 us to do almost all configuration in `manifest.yml` and skip modifying
 command line options in `runner/.profile` or having a .toml add on.
+
+## Testing
+
+To run the terraform tests:
+
+1. create a service account user with OrgManager permissions and set the `CF_USER` and `CF_PASSWORD` environment variables with that accounts details
+1. From the root directory, run `terraform init`
+1. Run `./bin/run_tests.sh`

bin/run_tests.sh

@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+# a little helper script to remind you to set up authentication and pass in the proper cf_user variable
+
+if [ -z "$CF_USER" ] || [ -z "$CF_PASSWORD" ]; then
+  echo "ERROR: Export CF_USER and CF_PASSWORD for a user with OrgManager permissions"
+  exit 1
+fi
+
+TF_VAR_cf_user="$CF_USER" terraform test

terraform/runner-manager/data.tf → data.tf

@@ -12,6 +12,6 @@ data "cloudfoundry_service_credential_binding" "runner-service-account-key" {
 # Archive a single file.
 data "archive_file" "src" {
   type        = "zip"
-  source_dir  = "${path.module}/../../runner-manager"
+  source_dir  = "${path.module}/runner-manager"
   output_path = "${path.module}/files/src.zip"
 }

terraform/runner-manager/main.tf → main.tf

@@ -9,6 +9,7 @@ locals {
     "*.fedoraproject.org",                 # fedora runner dependencies install
     "s3.dualstack.us-east-1.amazonaws.com" # gitlab-runner-helper source for runners
   ]
+  proxy_allowlist = setunion(local.devtools_egress_allowlist, var.worker_egress_allowlist)
 }
 
 # the `depends_on` lines for each resource or module is needed to properly sequence initial creation
@@ -168,7 +169,7 @@ module "egress_proxy" {
   cf_egress_space = module.egress_space.space
   name            = var.egress_app_name
   allowports      = [80, 443, 2222]
-  allowlist       = setunion(local.devtools_egress_allowlist, var.worker_egress_allowlist)
+  allowlist       = local.proxy_allowlist
   # see egress_proxy/variables.tf for full list of optional arguments
   depends_on = [module.egress_space]
 }

outputs.tf

@@ -0,0 +1,27 @@
+output "manager_app_id" {
+  value = cloudfoundry_app.gitlab-runner-manager.id
+}
+
+output "manager_space_id" {
+  value = module.manager_space.space_id
+}
+
+output "worker_space_id" {
+  value = module.worker_space.space_id
+}
+
+output "egress_space_id" {
+  value = module.egress_space.space_id
+}
+
+output "object_cache_service_id" {
+  value = module.object_store_instance.bucket_id
+}
+
+output "service_account_username" {
+  value = local.sa_cf_username
+}
+
+output "egress_app_id" {
+  value = module.egress_proxy.app_id
+}

providers.tf

@@ -0,0 +1,13 @@
+terraform {
+  required_version = "~> 1.5"
+  required_providers {
+    cloudfoundry = {
+      source  = "cloudfoundry/cloudfoundry"
+      version = ">= 1.1.0"
+    }
+    cloudfoundry-community = {
+      source  = "cloudfoundry-community/cloudfoundry"
+      version = ">= 0.53.1"
+    }
+  }
+}

terraform/create_service_account.sh → sandbox-deploy/create_service_account.sh

@@ -36,6 +36,7 @@ space=""
 service=""
 role="space-deployer"
 org_manager="false"
+org_manager_output=""
 
 while getopts ":hms:u:r:o:" opt; do
   case "$opt" in
@@ -52,6 +53,7 @@ while getopts ":hms:u:r:o:" opt; do
       org=${OPTARG}
       ;;
     m)
+      org_manager_output="-m"
       org_manager="true"
       ;;
     h)
@@ -84,7 +86,7 @@ if [[ $org_manager = "true" ]]; then
 fi
 
 cat << EOF
-# generated with $0 -s $space -u $service -r $role -o $org
+# generated with $0 -s $space -u $service -r $role -o $org $org_manager_output
 # revoke with $(dirname $0)/destroy_service_account.sh -s $space -u $service -o $org
 
 cf_user = "$username"

terraform/destroy_service_account.sh → sandbox-deploy/destroy_service_account.sh

@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 
-org="gsa-tts-benefits-studio"
+org="gsa-tts-devtools-prototyping"
 
 usage="
 $0: Destroy a Service User Account in a given space
@@ -46,8 +46,5 @@ fi
 
 cf target -o $org -s $space
 
-# destroy service key
-cf delete-service-key $service service-account-key -f
-
 # destroy service
 cf delete-service $service -f

terraform/runner-manager/providers.tf → sandbox-deploy/main.tf

@@ -11,15 +11,26 @@ terraform {
     }
   }
 }
-
 provider "cloudfoundry" {
   api_url  = "https://api.fr.cloud.gov"
   user     = var.cf_user
   password = var.cf_password
 }
-
 provider "cloudfoundry-community" {
   api_url  = "https://api.fr.cloud.gov"
   user     = var.cf_user
   password = var.cf_password
 }
+
+module "sandbox-runner" {
+  source = "../"
+
+  cf_user                 = var.cf_user
+  cf_space_prefix         = var.cf_space_prefix
+  ci_server_token         = var.ci_server_token
+  docker_hub_user         = var.docker_hub_user
+  docker_hub_token        = var.docker_hub_token
+  developer_emails        = var.developer_emails
+  worker_egress_allowlist = var.worker_egress_allowlist
+  allow_ssh               = var.allow_ssh
+}

sandbox-deploy/variables.tf

@@ -0,0 +1,21 @@
+variable "cf_user" {}
+variable "cf_password" {
+  sensitive = true
+}
+variable "cf_space_prefix" {}
+variable "ci_server_token" {
+  sensitive = true
+}
+variable "docker_hub_user" {}
+variable "docker_hub_token" {
+  sensitive = true
+}
+variable "developer_emails" {
+  type = list(string)
+}
+variable "worker_egress_allowlist" {
+  type = set(string)
+}
+variable "allow_ssh" {
+  type = bool
+}

sandbox-deploy/vars.tfvars-example

@@ -0,0 +1,9 @@
+allow_ssh        = true
+cf_space_prefix  = ""
+ci_server_token  = ""
+docker_hub_user  = ""
+docker_hub_token = ""
+developer_emails = []
+worker_egress_allowlist = [
+  "*.rubygems.org" # comment explaining why this is allowed
+]