diff --git a/src/cloud-api-adaptor/ibmcloud/README.md b/src/cloud-api-adaptor/ibmcloud/README.md index 5ebc192cf..08a1ca2f8 100644 --- a/src/cloud-api-adaptor/ibmcloud/README.md +++ b/src/cloud-api-adaptor/ibmcloud/README.md @@ -10,6 +10,9 @@ The high level flow involved is: - Deploy and validate that the nginx demo works - Clean-up and deprovision +For streamlined instructions to set up a simple demo for a Red Hat OpenShift Kubernetes cluster, refer to +[PeerPod setup using Red Hat OpenShift on IBM Cloud](./ROKS_SETUP.md). + ## Pre-reqs When building the peer pod VM image, it is simplest to use the container based approach, which only requires either @@ -76,8 +79,10 @@ needed in later steps. ## Create a 'self-managed' Kubernetes cluster on IBM Cloud provided infrastructure If you don't have a Kubernetes cluster for testing, you can follow the open-source -[instructions](./cluster) - to set up a basic cluster where the Kubernetes nodes run on IBM Cloud provided infrastructure. +[instructions](./cluster) to set up a basic cluster where the Kubernetes nodes run on IBM Cloud provided infrastructure. + +If you want to create, or use an existing, Red Hat OpenShift Kubernetes cluster, follow the instructions in +[Set up an OpenShift Kubernetes cluster for PeerPod VMs](./ROKS_SETUP.md#set-up-an-openshift-kubernetes-cluster-for-peerpod-vms). ## Deploy PeerPod Webhook @@ -94,25 +99,18 @@ If you don't have a Kubernetes cluster for testing, you can follow the open-sour #### Deploy the peer-pods webhook - From within the root directory of the `cloud-api-adaptor` repository, deploy the [webhook](../../webhook/) with: ``` - kubectl apply -f ./webhook/hack/webhook-deploy.yaml + kubectl apply -f ./src/webhook/hack/webhook-deploy.yaml ``` - Wait for the pods to all be in running state with: ``` kubectl get pods -n peer-pods-webhook-system --watch ``` -- Advertise the extended resource `kata.peerpods.io/vm.` by running the following commands: - ``` - pushd webhook/hack/extended-resources - ./setup.sh - popd - ``` - ## Deploy the Confidential-containers operator The `caa-provisioner-cli` simplifies deploying the operator and the cloud-api-adaptor resources on to any cluster. See the [test/tools/README.md](../test/tools/README.md) for full instructions. To create an ibmcloud ready version follow these steps ```bash -# Starting from the cloud-api-adaptor root directory +# Starting from directory src/cloud-api-adaptor of the cloud-api-adapter repository pushd test/tools make BUILTIN_CLOUD_PROVIDERS="ibmcloud" all popd diff --git a/src/cloud-api-adaptor/ibmcloud/ROKS_SETUP.md b/src/cloud-api-adaptor/ibmcloud/ROKS_SETUP.md new file mode 100644 index 000000000..4b0ca5ec5 --- /dev/null +++ b/src/cloud-api-adaptor/ibmcloud/ROKS_SETUP.md @@ -0,0 +1,288 @@ +# PeerPod setup using Red Hat OpenShift on IBM Cloud + +This guide describes how to set up a simple peer pod demo environment with a Red Hat OpenShift Kubernetes cluster on IBM Cloud. This involves the following steps: + +1. [Set up an OpenShift Kubernetes cluster for PeerPod VMs](#set-up-an-openshift-kubernetes-cluster-for-peerpod-vms) +1. [Upload a PeerPod VM Custom Image](#upload-a-peerpod-vm-custom-image) +1. [Deploy the PeerPod Webhook](#deploy-the-peerpod-webhook) +1. [Deploy the Confidential-containers operator](#deploy-the-confidential-containers-operator) +1. [Run a Helloworld sample](#run-a-helloworld-sample) + +## Pre-reqs + +Before proceeding you will need to install: + +1. the [Pre-reqs in README.md](./README.md#pre-reqs) but not including Terraform and Ansible, which are not used in this guide. +1. ibmcloud plugins: + - container-service[kubernetes-service/ks] + - vpc-infrastructure[infrastructure-service/is] +1. the OpenShift [oc CLI](https://cloud.ibm.com/docs/openshift?topic=openshift-cli-install#install-kubectl-cli) + +## Set up an OpenShift Kubernetes cluster for PeerPod VMs + +To set up a cluster for peer pod VMs, you need an IBM Cloud VPC, a subnet with a public gateway, and a COS instance. +Set the following environment variables to the values for your setup (note: make sure that your subnet has an attached public gateway): + +```bash +export IBMCLOUD_API_KEY= +export VPC_ID= +export SUBNET_ID= +export COS_CRN= +``` + +Log in to ibmcloud in the region corresponding to your subnet and then set zone and region environnment variables using the following commands: + +```bash +export ZONE="$(ibmcloud is subnet $SUBNET_ID -json | jq -r .zone.name)" +export REGION="$(ibmcloud is zone $ZONE -json | jq -r .region.name)" +``` + +If you have an existing OpenShift cluster, set the following variable to the name of your cluster. Otherwise set it to whatever name you'd like. + +```bash +export CLUSTER_NAME=kata-test-roks +``` + +### Create an Openshift cluster + +If you are using an existing cluster, you can skip this section and proceed to [configuring the cluster](#configure-an-openshift-cluster). + +1. Create a ROKS cluster + + ```bash + ibmcloud ks cluster create vpc-gen2 --flavor bx2.4x16 --name "$CLUSTER_NAME" --subnet-id "$SUBNET_ID" --vpc-id "$VPC_ID" --zone "$ZONE" --operating-system RHCOS --workers 2 --version 4.16.23_openshift --disable-outbound-traffic-protection --cos-instance "$COS_CRN" + + ``` + + Note: you will need to update the value of the `--version` argument to the current default version returned by `ibmcloud ks versions`, if it is different. + +1. Wait until the cluster is completely up and running before proceeding ... + +1. Get the `kubeconfig` for your cluster + + ```bash + ibmcloud ks cluster config --cluster "$CLUSTER_NAME" --admin + ``` + +### Configure an OpenShift cluster + +By default, your Red Hat OpenShift cluster will not work with the peer pod components. Using the environment variables set in the previous section, proceed with the following steps. + +1. Add a cluster inbound security group rule for the `kata-agent` client + + ```bash + export CLUSTER_ID=$(ibmcloud ks cluster get --cluster "$CLUSTER_NAME" --output json | jq -r .id) + export CLUSTER_SG="kube-$CLUSTER_ID" + export KATA_SG=$(ibmcloud is vpc "$VPC_ID" -json | jq -r .default_security_group.id) + ibmcloud is sg-rulec "$CLUSTER_SG" inbound udp --port-min 4789 --port-max 4789 --remote "$KATA_SG" + ``` + +1. Allow `cloud-api-adapter` to update pod finalizers + + ```bash + kubectl apply -n default -f - < with id is available`. Make note of the `image-id`, which will be +needed below. + +Note: If the import.sh script fails and the CLI has not been configured with the COS instance before, you will need to include the `--instance` argument. Refer to [IMPORT_PODVM_TO_VPC.md](https://github.com/confidential-containers/cloud-api-adaptor/blob/main/src/cloud-api-adaptor/ibmcloud/IMPORT_PODVM_TO_VPC.md#running) for details. + +## Deploy the PeerPod Webhook + +Follow the [webhook instructions in README.md](./README.md#deploy-peerpod-webhook) to deploy cert-manager and the peer-pods webhook. + +## Deploy the Confidential-containers operator + +The `caa-provisioner-cli` command can be use to simplify deployment of the operator and the cloud-api-adaptor resources on to any cluster. See the [test/tools/README.md](../test/tools/README.md) for full instructions. To create an ibmcloud-ready version of the provisioner CLI, run the following make command: + +```bash +# Starting from directory src/cloud-api-adaptor of the cloud-api-adapter repository +pushd test/tools +make BUILTIN_CLOUD_PROVIDERS="ibmcloud" all +popd +``` + +This will create `caa-provisioner-cli` in the `src/cloud-api-adaptor/test/tools` directory. To use the command you will need to set up a `.properties` file containing the relevant ibmcloud information to enable your cluster to create and use peer-pods. + +Set the SSH_KEY_ID and PODVM_IMAGE_ID environment variables and use the following command to generate the `.properties` file (Note that the IBMCLOUD_API_KEY, VPC_ID, and SUBNET_ID environment variables have already been set in [Set up an OpenShift Kubernetes cluster for PeerPod VMs +](#set-up-an-openshift-kubernetes-cluster-for-peerpod-vms)): + +```bash +export SSH_KEY_ID= # your ssh key id +export PODVM_IMAGE_ID= # the image id of the peerpod vm uploaded to ibmcloud +#export IBMCLOUD_API_KEY= # your ibmcloud apikey +#export VPC_ID= # vpc that the cluster is in +#export SUBNET_ID= # subnet to use (must have a public gateway attached) + +cat < ~/peerpods-cluster.properties +APIKEY="$IBMCLOUD_API_KEY" +SSH_KEY_ID="$SSH_KEY_ID" +PODVM_IMAGE_ID="$PODVM_IMAGE_ID" +VPC_ID="$VPC_ID" +VPC_SUBNET_ID="$SUBNET_ID" +VPC_SECURITY_GROUP_ID="$(ibmcloud is vpc "$VPC_ID" -json | jq -r .default_security_group.id)" +RESOURCE_GROUP_ID="$(ibmcloud is vpc "$VPC_ID" -json | jq -r .resource_group.id)" +ZONE="$(ibmcloud is subnet $SUBNET_ID -json | jq -r .zone.name)" +REGION="$(ibmcloud is zone $ZONE -json | jq -r .region.name)" +IBMCLOUD_PROVIDER="ibmcloud" +INSTANCE_PROFILE_NAME="bx2-2x8" +CAA_IMAGE_TAG="latest-amd64" +EOF +``` + +This will create a `peerpods-cluster.properties` files in your home directory. + +Finally, run the `caa-provisioner-cli` command to install the operator and cloud-api-adaptor: + +```bash +export CLOUD_PROVIDER=ibmcloud +export TEST_PROVISION_FILE="$HOME/peerpods-cluster.properties" +export TEST_TEARDOWN="no" +pushd test/tools +./caa-provisioner-cli -action=install +popd +``` + +Run the following command to confirm that the operator and cloud-api-adapter have been deployed: + +```bash +kubectl get pods -n confidential-containers-system +``` + +Once everything is up and ruuning, you should see output similar to the following: + +``` +cc-operator-controller-manager-7f8db55b55-r9thx 2/2 Running 0 7m55s +cc-operator-daemon-install-2dsnz 1/1 Running 0 6m59s +cc-operator-daemon-install-d522m 1/1 Running 0 6m58s +cc-operator-pre-install-daemon-gzl8w 1/1 Running 0 7m29s +cc-operator-pre-install-daemon-w5whl 1/1 Running 0 7m29s +cloud-api-adaptor-daemonset-m5x5s 1/1 Running 0 7m29s +cloud-api-adaptor-daemonset-v6jdr 1/1 Running 0 7m29s +peerpod-ctrl-controller-manager-65f76cb59-vhbt4 2/2 Running 0 5m20s +``` + +## Run a Helloworld sample + +You can run the following commands to validate that your cluster has been set up properly and is working as expected. + +```bash +kubectl apply -n default -f https://raw.githubusercontent.com/istio/istio/release-1.24/samples/curl/curl.yaml +kubectl apply -n default -f - <