Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add Cadvisor #225

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
110 changes: 110 additions & 0 deletions docs/services/cadvisor.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
# cAdvisor

This playbook can configure [cAdvisor](https://github.com/google/cadvisor)

## Dependencies

This service requires the following other services:

- (optionally) [Traefik](traefik.md) - a reverse-proxy server for exposing cadvisor publicly
- (optionally) [Prometheus](./prometheus.md) - a database for storing metrics
- (optionally) [Grafana](./grafana.md) - a web UI that can query the prometheus datasource (connection) and display the logs

## Configuration

To enable this service, add the following configuration to your `vars.yml` file and re-run the [installation](../installing.md) process:

```yaml
########################################################################
# #
# cadvisor #
# #
########################################################################

cadvisor_enabled: true
# You will have to mount specific folders depending on your need
# cadvisor_container_extra_arguments:
# - "--mount type=bind,source=/,destination=/rootfs,readonly"
# - "--mount type=bind,source=/sys,destination=/sys,readonly"
# - "--mount type=bind,source=/var/lib/docker/,destination=/var/lib/docker,readonly"
# - "--mount type=bind,source=/dev/disk/,destination=/dev/disk,readonly"

########################################################################
# #
# /cadvisor #
# #
########################################################################
```

See the full list of options in the [default/main.yml](default/main.yml) file

cAdvisor can scrape metrics from system and containers. These metrics can be :

- Displayed on the cAdvisor Web UI
- Exposed to a metric-storage server like [Prometheus](./prometheus.md).

## Exposing publicly cAdvisor

To expose cAdvisor WebUI and metrics to the web, you need to assign a hostname in `cadvisor_hostname` and optionally a path-prefix.

```yaml
# To expose the metrics publicly, enable and configure the lines below:
cadvisor_hostname: mash.example.com
cadvisor_path_prefix: /

# To protect the web ui and your metrics with HTTP Basic Auth, enable and configure the lines below.
# See: https://doc.traefik.io/traefik/middlewares/http/basicauth/#users
cadvisor_container_labels_traefik_middleware_basic_auth_enabled: true
cadvisor_container_labels_traefik_middleware_basic_auth_users: ""
```

## Integrating with Prometheus

### Prerequesites

The bare minimium is to ensure Prometheus can reach cadvisor.

- If cadvisor is on a different host than Prometheus, refer to section [Exposing publicly cAdvisor](cadvisor.md#Exposing-publicly-cAdvisor)
- If cadvisor is on the same host than prometheus, refer to section [Ensure Prometheus is on the same container network as cadvisor.](cadvisor.md#)

### Ensure Prometheus is on the same container network as cAdvisor.

If you are using MASH playbook, cadvisor should already be connected to prometheus container network

If cadvisor and prometheus do not share a network, you will have to

- Either connect Prometheus container network to cadvisor by editing `prometheus_container_additional_networks_auto`
- Either connect cadvisor container network to Prometheus by editing `cadvisor_container_additional_networks_custom`

Exemple:

```yaml
prometheus_container_additional_networks:
- "{{ cadvisor_container_network }}"
```

### Write the scrape config for prometheus

```yaml
prometheus_config_scrape_configs_additional:
- job_name: cadvisor
scrape_interval: 5s
scrape_timeout: 5s
static_configs:
- targets:
- "{{ cadvisor_identifier }}:8080"
```

replace the target by your ip_adress:port if cAdvisor is on a different host than Prometheus

# Healthcheck

```yaml
cadvisor_environment_variables_extension: |

# CADVISOR_HEALTHCHECK_URL=http://localhost:8080/healthz
```

## Usage

After [installing](../installing.md), refer to the documentation of [cAdvisor](https://github.com/google/cadvisor).
1 change: 1 addition & 0 deletions docs/supported-services.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
| [authentik](https://goauthentik.io/) | An open-source Identity Provider focused on flexibility and versatility. | [Link](services/authentik.md) |
| [borgbackup](https://www.borgbackup.org/) (via [borgmatic](https://torsion.org/borgmatic/)) | A deduplicating backup program with optional compression and encryption| [Link](services/backup-borg.md) |
| [Calibre-Web](https://github.com/janeczku/calibre-web) | Web app for browsing, reading and downloading eBooks stored in a [Calibre](https://calibre-ebook.com/) database | [Link](services/calibre-web.md) |
| [cAdvisor](https://github.com/google/cadvisor) | Analyzes resource usage and performance characteristics of running containers. | [Link](services/cadvisor.md) |
| [Changedetection.io](https://github.com/dgtlmoon/changedetection.io) | A simple website change detection and restock monitoring solution. | [Link](services/changedetection.md) |
| [ClickHouse](https://clickhouse.com/) | An open-source column-oriented DBMS for online analytical processing (OLAP) that allows users to generate analytical reports using SQL queries in real-time. | [Link](services/clickhouse.md) |
| [Collabora Online](https://www.collaboraoffice.com/) | Your Private Office Suite In The Cloud | [Link](services/collabora-online.md) |
Expand Down
54 changes: 54 additions & 0 deletions templates/group_vars_mash_servers
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,11 @@ mash_playbook_devture_systemd_service_manager_services_list_auto_itemized:
{{ ({'name': (authelia_identifier + '.service'), 'priority': 2000, 'groups': ['mash', 'authelia']} if authelia_enabled else omit) }}
# /role-specific:authelia

# role-specific:cadvisor
- |-
{{ ({'name': (cadvisor_identifier + '.service'), 'priority': 500, 'groups': ['mash', 'metrics', 'prometheus-node-exporter']} if cadvisor_enabled else omit) }}
sudo-Tiz marked this conversation as resolved.
Show resolved Hide resolved
# /role-specific:cadvisor

# role-specific:calibre-web
- |-
{{ ({'name': (calibre_web_identifier + '.service'), 'priority': 2000, 'groups': ['mash', 'calibre-web']} if calibre_web_enabled else omit) }}
Expand Down Expand Up @@ -1596,6 +1601,55 @@ wetty_container_labels_traefik_tls_certResolver: "{{ devture_traefik_certResolve



# role-specific:cadvisor
########################################################################
# #
# cadvisor #
# #
########################################################################

cadvisor_enabled: false

cadvisor_identifier: "{{ mash_playbook_service_identifier_prefix }}cadvisor"

cadvisor_uid: "{{ mash_playbook_uid }}"
cadvisor_gid: "{{ mash_playbook_gid }}"

# Only enable Traefik labels if a hostname is set (indicating that this will be exposed publicly)
sudo-Tiz marked this conversation as resolved.
Show resolved Hide resolved
cadvisor_container_labels_traefik_enabled: "{{ mash_playbook_traefik_labels_enabled and cadvisor_hostname }}"
cadvisor_container_labels_traefik_docker_network: "{{ mash_playbook_reverse_proxyable_services_additional_network }}"
cadvisor_container_labels_traefik_entrypoints: "{{ devture_traefik_entrypoint_primary }}"
cadvisor_container_labels_traefik_tls_certResolver: "{{ devture_traefik_certResolver_primary }}"

cadvisor_container_labels_metrics_enabled: "{{ prometheus_enabled | default(false) or mash_playbook_metrics_exposure_enabled }}"
cadvisor_container_labels_metrics_hostname: "{{ mash_playbook_metrics_exposure_hostname }}"
cadvisor_container_labels_metrics_path_prefix: "{{ mash_playbook_metrics_exposure_path_prefix }}/{{ cadvisor_identifier }}"
cadvisor_container_labels_metrics_traefik_middleware_basic_auth_enabled: "{{ mash_playbook_metrics_exposure_http_basic_auth_enabled }}"
cadvisor_container_labels_metrics_traefik_middleware_basic_auth_users: "{{ mash_playbook_metrics_exposure_http_basic_auth_users }}"
cadvisor_container_labels_metrics_middleware_basic_auth_enabled: "{{ mash_playbook_metrics_exposure_http_basic_auth_enabled }}"
cadvisor_container_labels_metrics_middleware_basic_auth_users: "{{ mash_playbook_metrics_exposure_http_basic_auth_users }}"
Comment on lines +1692 to +1698
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These variables do not seem to be defined anymore, yet.. they're here.

That said, I think it's better if metrics had their own Traefik router (separate from the web UI) and for them to respect the mash_playbook_metrics_exposure_* variables automatically (auto-enabling metrics exposure for this service, possibly protected with the Basic Auth credentials specified in mash_playbook_metrics_exposure_http_basic_auth_*).

The web UI could remain optional and have its (optional) separate set of Basic Auth credentials


cadvisor_container_additional_networks_auto: |
{{
(
([mash_playbook_reverse_proxyable_services_additional_network] if mash_playbook_reverse_proxyable_services_additional_network else [])
+
([prometheus_container_network] if (prometheus_enabled | default(false) and prometheus_container_network | default('') != cadvisor_container_network) else [])
sudo-Tiz marked this conversation as resolved.
Show resolved Hide resolved
) | unique
}}

cadvisor_process_extra_arguments_auto:
- "--url_base_prefix={{ cadvisor_container_labels_traefik_path_prefix }}"
sudo-Tiz marked this conversation as resolved.
Show resolved Hide resolved

########################################################################
# #
# /cadvisor #
# #
########################################################################
# /role-specific:cadvisor



# role-specific:calibre-web
########################################################################
# #
Expand Down
4 changes: 4 additions & 0 deletions templates/requirements.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,10 @@
version: v1.2.8-1.8.12-1
name: backup_borg
activation_prefix: backup_borg_
- src: git+https://github.com/sudo-Tiz/ansible-role-cadvisor.git
version: main
name: cadvisor
activation_prefix: cadvisor_
- src: git+https://github.com/lingawakad/ansible-role-calibre-web.git
version: v0.6.21-1
name: calibre-web
Expand Down
4 changes: 4 additions & 0 deletions templates/setup.yml
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,10 @@
- role: galaxy/calibre-web
# /role-specific:calibre-web

# role-specific:cadvisor
- role: galaxy/cadvisor
# /role-specific:cadvisor

# role-specific:clickhouse
- role: galaxy/clickhouse
# /role-specific:clickhouse
Expand Down