diff --git a/README.md b/README.md index 865e1d92..e745370b 100644 --- a/README.md +++ b/README.md @@ -4,15 +4,15 @@ Maintainers: [Heptio](https://github.com/heptio) ## Overview -Heptio Gimbal is a layer-7 load balancing platform built on Kubernetes, Envoy, and Contour. It provides a scalable, multi-team, and API-driven ingress tier capable of routing internet traffic to multiple upstream Kubernetes clusters and traditional infrastructure technologies including OpenStack. +Heptio Gimbal is a layer-7 load balancing platform built on Kubernetes, the [Envoy proxy](https://www.envoyproxy.io/), and Heptio's Kubernetes Ingress controller, [Contour](https://heptio.github.io/contour/). It provides a scalable, multi-team, and API-driven ingress tier capable of routing Internet traffic to multiple upstream Kubernetes clusters and to traditional infrastructure technologies such as OpenStack. -Gimbal was developed out of a joint effort between Heptio and Yahoo Japan Corporation's subsidiary, Actapio, to modernize Yahoo Japan’s infrastructure with Kubernetes without impacting legacy investments in OpenStack. +Gimbal was developed out of a joint effort between Heptio and Yahoo Japan Corporation's subsidiary, Actapio, to modernize Yahoo Japan’s infrastructure with Kubernetes, without affecting legacy investments in OpenStack. At launch, Gimbal can discover services from Kubernetes and OpenStack clusters, but we expect to support additional platforms in the future. ### Common Use Cases -* Organizations with multiple Kubernetes clusters that need a way to manage ingress traffic across them +* Organizations with multiple Kubernetes clusters that need a way to manage ingress traffic across clusters * Organizations with Kubernetes and OpenStack infrastructure that need a consistent load balancing tier * Organizations that want to enable their development teams to safely self-manage their routing configuration * Organizations with bare metal or on-premises infrastructure that want cloud-like load balancing capabilities @@ -21,25 +21,25 @@ At launch, Gimbal can discover services from Kubernetes and OpenStack clusters, ## Prerequisites -Gimbal is tested with Kubernetes clusters running version 1.9 and later but should work with any cluster starting at version 1.7. +Gimbal is tested with Kubernetes clusters running version 1.9 and later but should work with any cluster running version 1.7 or later. Gimbal's service discovery is currently tested with Kubernetes 1.7+ and OpenStack Mitaka. ## Get started -Deployment of Gimbal is outlined in the [deployment section](deployment/README.md) and also includes quick start applications. +Deployment of Gimbal is outlined in the [deployment section](deployment/README.md), which includes quick start applications. ## Documentation -Documentation on all the Gimbal components can be found on the [docs page](docs/README.md). +Documentation for all the Gimbal components can be found in the [docs directory](docs/README.md). ## Known Limitations * Upstream Kubernetes Pods and OpenStack VMs must be routable from the Gimbal load balancing cluster - * No support for Kubernetes clusters with Overlay networks + * No support for Kubernetes clusters with overlay networks * We are looking for feedback on community requirements to design a solution. One potential option is to use one GRE tunnel per upstream cluster. [Feedback welcome here](https://github.com/heptio/gimbal/issues/39)! -* Kubernetes Ingress API is limited and insecure +* The Kubernetes Ingress API is limited and insecure * Only one backend per route * Anyone can modify route rules for a domain * More complex load balancing features like weighting and strategy are not supported @@ -47,7 +47,7 @@ Documentation on all the Gimbal components can be found on the [docs page](docs/ ## Troubleshooting -If you encounter any problems that the documentation does not address, please [file an issue](https://github.com/heptio/gimbal/issues) or talk to us on the Kubernetes Slack team channel `#gimbal`. +If you encounter any problems that the documentation does not address, please [file an issue](https://github.com/heptio/gimbal/issues) or talk to us on the Kubernetes Slack team channel #gimbal. ## Contributing diff --git a/deployment/README.md b/deployment/README.md index f07b4742..fb7d643a 100644 --- a/deployment/README.md +++ b/deployment/README.md @@ -1,46 +1,42 @@ +# Deployment -- [Deployment](#deployment) - - [Setup / Requirements](#setup--requirements) - - [Contour](#contour) - - [Discoverers](#discoverers) - - [Kubernetes](#kubernetes) - - [Openstack](#openstack) - - [Prometheus](#prometheus) - - [Quick start](#quick-start) - - [Access the Prometheus Web UI](#access-the-prometheus-web-ui) - - [Access the Alert Manager Web UI](#access-the-alert-manager-web-ui) - - [Grafana](#grafana) - - [Quick Start](#quick-start) - - [Accessing Grafana UI](#accessing-grafana-ui) - - [Configure Grafana](#configure-grafana) - - [Configure Datasource](#configure-datasource) - - [Dashboards](#dashboards) - - [Add Sample Kubernetes Dashboard](#add-sample-kubernetes-dashboard) - - [Validation](#validation) - - [Discovery Cluster](#discovery-cluster) - - [Gimbal Cluster](#gimbal-cluster) +- [Prerequisites](#prerequisites) +- [Deploy Contour](#deploy-contour) +- [Deploy Discoverers](#discoverers) + - [Kubernetes](#kubernetes) + - [Openstack](#openstack) +- [Deploy Prometheus](#deploy-prometheus) + - [Quick start](#quick-start) + - [Access the Prometheus web UI](#access-the-prometheus-web-ui) + - [Access the Alertmanager web UI](#access-the-alertmanager-web-ui) +- [Deploy Grafana](#deploy-grafana) + - [Quick start](#quick-start) + - [Access Grafana UI](#access-grafana-ui) + - [Configure Grafana](#configure-grafana) + - [Configure datasource](#configure-datasource) + - [Dashboards](#dashboards) + - [Add Sample Kubernetes Dashboard](#add-sample-kubernetes-dashboard) +- [Validation](#validation) + - [Discovery cluster](#discovery-cluster) + - [Gimbal cluster](#gimbal-cluster) -# Deployment -Following are instructions to get all the components up and running. +## Prerequisites -## Setup / Requirements +- A copy of this repository. Download, or clone: -A copy of this repo locally which is easily accomplished by cloning or downloading a copy: + ```sh + $ git clone git@github.com:heptio/gimbal.git + ``` -```sh -$ git clone git@github.com:heptio/gimbal.git -``` +- A single Kubernetes cluster to deploy Gimbal +- Kubernetes or Openstack clusters with flat networking. That is, each Pod has a route-able IP address on the network. -Additionally, this guide will assume a single Kubernetes cluster to deploy Gimbal, as well as Kubernetes or Openstack clusters with `flat` networking, meaning, pods get route-able IP address on the network. +## Deploy Contour -## Contour - -Contour is the system which handles the Ingress traffic. It utilizes Envoy which is an L7 proxy and communication bus designed for large modern service oriented architectures. - -Envoy is the data component of Contour and handles the network traffic, Contour drives the configuration of Envoy based upon the Kubernetes cluster configuration. +For information about Contour, see [the Gimbal architecture doc](../docs/gimbal-architecture.md). ```sh # Navigate to deployment directory @@ -50,11 +46,11 @@ $ cd deployment $ kubectl create -f contour/ ``` -_NOTE: The current configuration exposes the Envoy Admin UI so that Prometheus can scrape for metrics!_ +**NOTE**: The current configuration exposes the Envoy Admin UI so that Prometheus can scrape for metrics. -## Discoverers +## Deploy Discoverers -Service discovery is enabled via the Discoverers which have both Kubernetes and Openstack implementations. +Service discovery is enabled with the Discoverers, which have both Kubernetes and Openstack implementations. ```sh # Create gimbal-discoverer namespace @@ -65,7 +61,7 @@ kubectl create -f gimbal-discoverer/01-common.yaml The Kubernetes Discoverer is responsible for looking at all services and endpoints in a Kubernetes cluster and synchronizing them to the host cluster. -[Credentials](../docs/kubernetes-discoverer.md#credentials) to the remote cluster are required to be created as a secret. +[Credentials](../docs/kubernetes-discoverer.md#credentials) to the remote cluster must be created as a Secret. ```sh # Kubernetes secret @@ -77,13 +73,13 @@ $ kubectl -n gimbal-discovery create secret generic remote-discover-kubecfg \ $ kubectl apply -f gimbal-discoverer/02-kubernetes-discoverer.yaml ``` -Technical details on how the Kubernetes Discoverer works can be found in the [docs section](../docs/kubernetes-discoverer.md). +For more information, see [the Kubenetes Discoverer doc](../docs/kubernetes-discoverer.md). ### Openstack The Openstack Discoverer is responsible for looking at all LBaSS and members in an Openstack cluster and synchronizing them to the host cluster. -[Credentials](../docs/openstack-discoverer.md#credentials) to the remote cluster are required to be created as a secret. +[Credentials](../docs/openstack-discoverer.md#credentials) to the remote cluster must be created as a secret. ```sh # Openstack secret @@ -99,11 +95,11 @@ $ kubectl -n gimbal-discovery create secret generic remote-discover-openstack \ $ kubectl apply -f gimbal-discoverer/02-openstack-discoverer.yaml ``` -Technical details on how the Openstack Discoverer works can be found in the [docs section](../docs/openstack-discoverer.md). +For more information, see [the OpenStack Discoverer doc](../docs/openstack-discoverer.md). ## Prometheus -This directory contains a sample development deployment of Prometheus and Alert Manager using temporary storage (e.g. emptyDir space). +Sample development deployment of Prometheus and Alertmanager using temporary storage. ### Quick start @@ -117,7 +113,7 @@ $ cd kube-state-metrics $ kubectl apply -f kubernetes/ ``` -### Access the Prometheus Web UI +### Access the Prometheus web UI ```sh $ kubectl -n gimbal-monitoring port-forward $(kubectl -n gimbal-monitoring get pods -l app=prometheus -l component=server -o jsonpath='{.items[0].metadata.name}') 9090:9090 @@ -125,7 +121,7 @@ $ kubectl -n gimbal-monitoring port-forward $(kubectl -n gimbal-monitoring get p then go to [http://localhost:9090](http://localhost:9090) in your browser -### Access the Alert Manager Web UI +### Access the Alertmanager web UI ```sh $ kubectl -n gimbal-monitoring port-forward $(kubectl -n gimbal-monitoring get pods -l app=prometheus -l component=alertmanager -o jsonpath='{.items[0].metadata.name}') 9093:9093 @@ -137,7 +133,7 @@ then go to [http://localhost:9093](http://localhost:9093) in your browser Sample development deployment of Grafana using temporary storage. -### Quick Start +### Quick start ```sh # Deploy @@ -149,26 +145,26 @@ $ kubectl create secret generic grafana -n gimbal-monitoring \ --from-literal=grafana-admin-user=admin ``` -### Accessing Grafana UI +### Access the Grafana UI ```sh $ kubectl port-forward $(kubectl get pods -l app=grafana -n gimbal-monitoring -o jsonpath='{.items[0].metadata.name}') 3000 -n gimbal-monitoring ``` -Access Grafana at http://localhost:3000 in your browser. Use `admin` as the username. +then go to [http://localhost:3000](http://localhost:3000) in your browser, with `admin` as the username. ### Configure Grafana -Grafana requires some configuration after it's deployed, use the following steps to configure a datasource and import a dashboard to validate the connection. +Grafana requires some configuration after it's deployed. These steps configure a datasource and import a dashboard to validate the connection. -#### Configure Datasource +#### Configure datasource -1. From main Grafana page, click on `Add Datasource` -2. For `Name` enter `prometheus` -3. Choose `Prometheus` under `Type` selector -4. In the next section, enter `http://prometheus:9090` for the `URL` -5. Click `Save & Test` -6. Look for the message box in green stating `Data source is working` +1. On the main Grafana page, click **Add Datasource** +2. For **Name** enter _prometheus_ +3. In `Type` selector, choose _Prometheus_ +4. For the URL, enter `http://prometheus:9090` +5. Click **Save & Test** +6. Look for the message box in green stating _Data source is working_ #### Dashboards @@ -176,29 +172,29 @@ Dashboards for Envoy and the Discovery components are included as part of the sa ##### Add Sample Kubernetes Dashboard -Add sample dashboard to validate data source is collecting data: +Add sample dashboard to validate that the data source is collecting data: -1. From main page, click on `plus` icon and choose `Import dashboard` -2. Enter `1621` in the first box -3. Under the `prometheus` section choose the data source created in previous section -4. Click `Import` +1. On the main page, click the plus icon and choose **Import dashboard** +2. Enter _1621_ in the first box +3. In the **prometheus** section, choose the datasource that you just created +4. Click **Import** ## Validation -Once the components are deployed, the deployment can be verified with the following steps: +Now you can verify the deployment: -### Discovery Cluster +### Discovery cluster -This example utilizes a Kubernetes cluster as the discovered cluster which was configured [previously](#kubernetes). We will deploy a few sample applications into the `default` namespace: +This example deploys a sample application into the default namespace of [the discovered Kubernetes cluster that you created](#kubernetes). ```sh # Deploy sample apps $ kubectl apply -f example-workload/deployment.yaml ``` -### Gimbal Cluster +### Gimbal cluster -These commands should be run on the Gimbal cluster to verify it's components: +Run the following commands on the Gimbal cluster to verify its components: ```sh # Verify Discoverer Components diff --git a/docs/README.md b/docs/README.md index a36e8628..97c9fbff 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,25 +1,16 @@ # Documentation -Heptio Gimbal is a layer-7 load balancing platform built on Kubernetes, Envoy, and Contour. It provides a scalable, multi-team, and API-driven ingress tier capable of routing internet traffic to multiple upstream Kubernetes clusters and traditional infrastructure technologies including OpenStack. +See the root-level README for an introduction, and the [deployment directory](../deployment/README.md) to get started with setting up and and deploying Gimbal. -## Data Flow - -![Data Flow](images/data-flow.png) - -## Architecture Overview - -![Arch Overview](images/overview.png) - -You can read more about the [Gimbal Architecture](gimbal-architecture.md). +Here you can dig into the details of how Gimbal works, and explore more advanced topics for operators and users. ## Overview Guides -The following guides will describe how components of Gimbal function and interact with other systems: +These guides describe how the components of Gimbal function and how they interact with other systems: + - [Kubernetes Discoverer](kubernetes-discoverer.md) - [Openstack Discoverer](openstack-discoverer.md) -Guides on how to setup / deploy Gimbal can be found in the [deployment guide](../deployment/README.md). - ## Operator Topics - [Manage Backend Clusters and Discovery](manage-backends.md) @@ -31,3 +22,13 @@ Guides on how to setup / deploy Gimbal can be found in the [deployment guide](.. - [Route Specification](route.md) - [Dashboards / Monitoring / Alerting](monitoring.md) + +## Data Flow + +![Data Flow](images/data-flow.png) + +## Architecture Overview + +![Arch Overview](images/overview.png) + +More about the [Gimbal Architecture](gimbal-architecture.md). diff --git a/docs/gimbal-architecture.md b/docs/gimbal-architecture.md index 79988705..43cedda2 100644 --- a/docs/gimbal-architecture.md +++ b/docs/gimbal-architecture.md @@ -6,48 +6,48 @@ Gimbal is designed to be deployed to one or more Kubernetes clusters that will a ![Gimbal Architecture](images/gimbal-arch.png) -## Gimbal Load Balancing Deployment +## Gimbal Deployment Cluster administrators deploy Gimbal and its dependencies to the appropriate namespaces. -* Gimbal service discovery agents (one per upstream cluster) run in the `gimbal-discovery` namespace +* Gimbal Service discovery agents (one per upstream cluster) run in the `gimbal-discovery` namespace * Contour and Envoy run in the `gimbal-contour` namespace -* The optional monitoring suite runs in the `gimbal-monitoring` namespace. +* The optional monitoring suite runs in the `gimbal-monitoring` namespace ![Arch 01](images/overview.png) -## Gimbal Service Discovery +## Gimbal Service discovery -Once deployed, the service discoverers continuously collect information about upstream applications running in the Kubernetes or OpenStack clusters and create corresponding `Service` and `Endpoint` objects in the appropriate team namespaces. +After they are deployed, the Service Discoverers continuously collect information about upstream applications running in the Kubernetes or OpenStack clusters and create corresponding Service and Endpoint objects in the appropriate team namespaces. -For example, assuming there is namespace in `Kubernetes Cluster A` called `app-team-1`, any Services and Endpoints discovered will be replicated in the Gimbal cluster within the `app-team-1` namespace. Labels associated with the services are replicated as well. +For example, let's assume a namespace in Kubernetes Cluster A called `app-team-1`. Any Services and Endpoints that are discovered in Cluster A are replicated in the Gimbal cluster in the `app-team-1` namespace. Labels associated with the Services are also replicated. -The OpenStack discoverer provides similar behavior by monitoring all Load Balancers as a Service (LBaaS) configured as well as the corresponding Members. They are synchronized to the team's namespace as Services and Endpoints, with the namespace being configured as the TenantName in OpenStack. +The OpenStack Discoverer provides similar behavior by monitoring all load balancers and their members, using LBaaS. The OpenStack load balancers are translated into Services, and the pool members are translated into Endpoints. The OpenStack tenant name becomes the cluster namespace. ![Arch 02](images/arch-01-service.png) -## Multi-team Route Configuration +## Multi-team route configuration -Development teams can see which Services are available to them by using standard Kubernetes tools like `kubectl`. Services discovered by Gimbal are augmented with additional labels including the name of the cluster they were discovered which enables querying using selectors. +Development teams can see which Services are available to them by using standard Kubernetes tools like kubectl. Services discovered by Gimbal are augmented with additional labels including the name of the cluster they were discovered which enables querying using selectors. -Developers create Kubernetes Ingress objects which define where inbound traffic (e.g. myapp.company.com) is routed. +Developers create Kubernetes Ingress objects that define where inbound traffic (e.g. myapp.company.com) is routed. ![Arch 03](images/arch-02-route.png) ## Contour -Contour is a Kubernetes Ingress Controller for Envoy that continuously monitors Ingress, Service, and Endpoint objects in the team namespaces. +Contour is a Kubernetes Ingress controller for the Envoy proxy that continuously monitors Ingress, Service, and Endpoint objects in the team namespaces. ![Arch 04](images/arch-03-contour.png) -Contour provides an Envoy API compatible GRPC endpoint which will dynamically modify the Envoy route configuration. +Contour provides an Envoy API compatible gRPC endpoint that dynamically modifies the Envoy route configuration. ![Arch 04](images/arch-04-envoy.png) -Envoy, deployed using the HostNetwork, provides the data plane for Gimbal. Envoy is a high-performance load balancing proxy that physically routes ingress traffic to the upstream Kubernetes and OpenStack clusters. +Envoy is deployed using the host network, and provides the data plane for Gimbal. Envoy is a high-performance load balancing proxy that physically routes ingress traffic to the upstream Kubernetes and OpenStack clusters. ## Monitoring -Gimbal includes an optional monitoring add-on that includes Prometheus, AlertManager, and Grafana. +Gimbal includes an optional monitoring addon that includes Prometheus, Alertmanager, and Grafana. Each Gimbal system component exposes a Prometheus-compatible metrics endpoint with health status and essential metrics that are aggregated by Prometheus and can be visualized using Grafana. \ No newline at end of file