Fix READ.ME & update descriptions

* Update documentation * Fixes READ.ME instructions for module deployment * Updates docs/ARCHITECTURE.md * Updates docs/SETUP.md Signed-off-by: Nindemic <[email protected]>
smaddis · Dec 13, 2021 · eb1aebc · eb1aebc
1 parent 731e1e6
commit eb1aebc
Show file tree

Hide file tree

Showing 3 changed files with 86 additions and 38 deletions.
diff --git a/README.md b/README.md
@@ -21,29 +21,46 @@ Architectural description of the codebase can be found at [ARCHITECTURE.md](./do
 
 ## Usage
 
-1. Create Terraform State storage group and account to Azure
+### 1. Create Terraform State storage group and account to Azure
+
 ```bash
+$ terraform -chdir=00_tfstate_storage init 
 $ terraform apply ./00_tfstate_storage
 ```
-### No separate storage resource group (default)
 
-2. Remember to edit `main.tf` email variable to a real one for TLS certificate
-2. Deploy main service stack
+### 2. Create separate resource group for persistent data
 
 ```bash
-$ terraform apply ./
+$ cd 01_storage_rg
+$ terraform init
 ```
+2. Create a Terraform workspace
 
-### OPTIONAL: Separate resource group
-
-2. Create separate resource group for databases
+```bash
+$ terraform workspace new [WORKSPACE NAME]
 ```
+
+2. Deploy persistent data file shares
+
+```bash
+$ cd ../
 $ terraform apply ./01_storage_rg
 ```
 
-3. Deploy main service stack
+### 3. Deploy main service stack
+
+Remember to edit `main.tf` email variable to a real one for TLS certificate
+
+```bash
+$ cd 02_deployHono
+$ terraform init
 ```
-$ terraform apply ./02_deployHono
+
+Create a Terraform workspace. Important: Use same workspace name as before in 01_storage_rg!
+
+```bash
+$ terraform workspace new [WORKSPACE NAME]
+$ terraform apply
 ```
 
 ## After deployment

diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md
@@ -6,19 +6,18 @@ This is a architectural description of the smad-deploy-azure
 |------|----------|-------|
 |[./00_tfstate_storage](#Terraform-state-module)|Creates resource group for terraform state file|
 |[./01_storage_rg](#Storage-resource-group)|Creates separate resource group for persistent data needs|
-|[./02_container_registry](#Contrainer-registry-module)| Creates ACR for k8s cluster. **Currently not used**|
 |[./02_deployHono](#Hono-service-stack-module)| Creates Hono k8s cluster and services
 |.02_deployHono/modules|Modules used by the script. |
 |[.../k8s](#Kubernetes-deployment-module)|Module for  creating kubernetes cluster to Azure (AKS)
 |[.../ambassador](#Ambassador-deployment-module)|Handles deployment of Ambassador via Helm to k8s cluster | k8s
 |[.../hono](#Hono-deployment-module)|Handles deployment of Hono via Helm to k8s cluster | k8s
 |[../influxdb](#Influxdb-module)|Module that handles deployment of Influxdb to k8s cluster. Holds all the information to set up database for prometheus metrics | k8s
 |[.../jaeger](#Jaeger-deployment-module)|Handles deployment of Jaeger via Helm to k8s cluster | k8s
+|[.../kafka](#Kafka-deployment-module)|Handles deployment of Kafka cluster via Helm to k8s cluster | k8s
 |[.../kube_prometheus_stack](#kube-prometheus-stack-deployment-module)|Handles deployment of kube-prometheus-stack via Helm to k8s cluster. | k8s
 |[.../mongodb](#Mongo-deployment-module)|Handles deployment of MongoDB via Helm to k8s cluster. | k8s
-
-
-Every module follows the conventional Terraform naming scheme, and therefore has `main.tf`, `variables.tf` and `outputs.tf` files.
+|[.../persistent_storage](#Data-persistency-module)|Deploys persistent volumes and persistent volume claims. | k8s
+|[./03_container_registry](#Contrainer-registry-module)| Creates ACR for k8s cluster. **Currently not used**|
 
 ## Description
 
@@ -37,7 +36,7 @@ Hold variables for naming resources created by this module.
 
 Output values four resource group, storage account and storate container.
 
-## Storage resource group module
+## Storage resource group
 
 This modules creates separate resource group for persistent volume claim.
 
@@ -51,23 +50,6 @@ Establish azurerm backend with previously set naming for tfstate files, and crea
 
 Output value of created resource group's id.
 
-## Contrainer registry module
-
-**NOT USED**
-
-### `main.tf`
-
-Creates Azure Container registry in the same resource group as k8s modules.
-
-Assigns acrpull role for k8s cluster
-### `variables.tf`
-
-Variables for naming resources.
-
-### `outputs.tf`
-
-Output values for ACR. Containing id, login url, username and password.
-
 
 ## Hono service stack module
 
@@ -109,6 +91,8 @@ Adds helm charts that bootstrap an Ambassador deployment on the k8s cluster usin
 #### `module "jaeger"`
 Adds helm charts that bootstrap a Jaeger deployment on the k8s cluster using the Helm package manager.
 
+#### `module "kafka"`
+Adds helm charts that bootstrap a Kafka cluster on the k8s cluster using the Helm package manager.
 
 ### `variables.tf`
 
@@ -297,3 +281,20 @@ Deploys jaeger-operator, and is configured with values from `jaeger_values.yaml`
 ### `values.yaml`
 
 Jaeger is enabled with simple metadata.
+
+## Contrainer registry module
+
+**NOT USED**
+
+### `main.tf`
+
+Creates Azure Container registry in the same resource group as k8s modules.
+
+Assigns acrpull role for k8s cluster
+### `variables.tf`
+
+Variables for naming resources.
+
+### `outputs.tf`
+
+Output values for ACR. Containing id, login url, username and password.
diff --git a/docs/SETUP.md b/docs/SETUP.md
@@ -52,10 +52,38 @@ To change your selected subscription
 $ az account set -s <$SUBSCRIPTION_ID_OR_NAME>
 ```
 
+# Cloud deployment
+
+Cloud deployment has four phases. These phases are represented as modules:
+00_tfstate_storage
+01_storage_rg
+02_deployHono
+03_container_registry *Currently not in use*
+
+### Initialising modules
+
+Modules that are in use need to be initialised before use with
+```
+$ terraform init
+```
+You need to be in the same directory as the module or use
+```
+$ terraform -chdir=[MODULE NAME] init 
+```
+ie.
+```
+$ terraform -chdir=00_tfstate_storage init 
+```
+#### Important
+Note that when running "$ terraform -chdir=[MODULE NAME]" default workspace is used. Do not use default workspace for modules other than 00_tfstate_storage. 
+
+Use same workspace name when deploying 01_storage_rg and 02_deployHono.
+See more about workspaces below.
+
 # 0. Create a storage account to store shared state for Terraform
 [Shared state](https://www.terraform.io/docs/language/state/remote.html) should always be preferred when working with Terraform.
 
-In order to create one run 
+In order to create one run after initializing 00_tfstate_storage module
 ```
 $ terraform apply './00_tfstate_storage'
 ```
@@ -111,7 +139,8 @@ Run
 ```
 $ terraform apply -var=use_separate_storage_rg=true
 ``` 
-to set a variable named 'use_separate_storage_rg' to have value 'true' via command line. With this variable set as 'true', you effectively switch the K8S cluster to use a separate resource group for storage.
+to set a variable named 'use_separate_storage_rg' to have value 'true' via command line. With this variable set as 'true', you effectively switch the K8S cluster to use a separate resource group for storage. 
+
 
 ### Terraform plan
 
@@ -125,7 +154,7 @@ Ensure that you have created proper Terraform state resources to Azure with [00_
 
 ## 1. Create separate resource group
 
-Create separate resource group for databases. Be sure to use the same workspace as when deploying the main service stack
+Create separate resource group for storing data. Be sure to use the same workspace as when deploying the main service stack
 
 ```
 $ terraform apply ./01_storage_rg
@@ -167,6 +196,7 @@ Similarly, when running `terraform apply -target`, if resources that are needed
 
 [`tests/honoscript`](../tests/honoscript) folder has a shell script that can be used to quickly verify that Hono is running properly. Refer to [`tests/honoscript/README.md`](../tests/honoscript/README.md) for more detailed instructions.
 
+<!--
 # Known issues
 
 ## Delays with Hono resource deletion
@@ -177,17 +207,17 @@ Workarounds:
 - Wait around 30-60 minutes before deploying Hono again.
 - Destroy the whole cluster (effectively the whole stack) and deploy it again (may be faster than the first option but you may lose some persistent data if not using the separate storage resource group).
 
-## Storage persistence
+ ## Storage persistence
 
 If the whole service stack is destroyed and variable `use_separate_storage_rg` is `false` all peristent volumes will also be destroyed. This can be prevented by setting `use_separate_storage_rg` to `true`: when the service stack is destroyed, the persistent volumes will remain in the separate resource group. Currently this has some drawbacks: when the service stack is deployed again, the script will create a new persistent volume and data from the old persistent volume must be restored manually.
 
 Workarounds:
-- No workarounds currently.
+- No workarounds currently. 
 
 ## Separate storage resource group access delay
 
 If the separate resource group for storage is used, a role will be created that grants the K8S cluster rights to access the separate resource group. It may take [up to 30 minutes](https://docs.microsoft.com/en-us/azure/role-based-access-control/troubleshooting#role-assignment-changes-are-not-being-detected) for Azure to propagate the permissions throughout the Azure subscription. Consequently, the K8S cluster may not be able to create the Persistent Volume Claim it needs for persistent data (e.g. for Hono device registry).
 
 Workarounds:
 - Run `$ terraform apply` again until the deployment succeeds.
-
+-->