diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 000000000000..6313b56c5784 --- /dev/null +++ b/.gitattributes @@ -0,0 +1 @@ +* text=auto eol=lf diff --git a/.template-helpers/arches.sh b/.template-helpers/arches.sh index 4baf2fd659b6..9d2765820622 100755 --- a/.template-helpers/arches.sh +++ b/.template-helpers/arches.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/opt/homebrew/bin/bash set -Eeuo pipefail repo="${1:-}" diff --git a/.template-helpers/generate-dockerfile-links-partial.sh b/.template-helpers/generate-dockerfile-links-partial.sh index 126fcbea3920..2b04d5357744 100755 --- a/.template-helpers/generate-dockerfile-links-partial.sh +++ b/.template-helpers/generate-dockerfile-links-partial.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/opt/homebrew/bin/bash set -Eeuo pipefail repo="${1:-}" diff --git a/.template-helpers/variant.sh b/.template-helpers/variant.sh index dd802ec5c82d..72bfafad3e96 100755 --- a/.template-helpers/variant.sh +++ b/.template-helpers/variant.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/opt/homebrew/bin/bash set -eo pipefail repo="$1" diff --git a/arangodb/content.md b/arangodb/content.md index 159a5569c66f..2cc2b4a207b0 100644 --- a/arangodb/content.md +++ b/arangodb/content.md @@ -49,11 +49,11 @@ docker inspect --format '{{ .NetworkSettings.IPAddress }}' arangodb-instance When using Docker, you need to specify the language you want to initialize the server to on the first run in one of the following ways: -- Set the environment variable `LANG` to a locale in the `docker run` command, e.g. `-e LANG=sv` for a Swedish locale. +- From v3.12.2 onward: Append the `--icu-language` startup option to the end of the Docker command, like `docker run ... %%IMAGE%% --icu-language sv` for a Swedish locale. -- Use an `arangod.conf` configuration file that sets a language and mount it into the container. For example, create a configuration file on your host system in which you set `icu-language = sv` at the top (before any `[section]`) and then mount the file over the default configuration file like `docker run -v /your-host-path/arangod.conf:/etc/arangodb3/arangod.conf ...`. +- Set the environment variable `LANG` to a locale in the `docker run` command, e.g. `-e LANG=sv`. -Note that you cannot set the language using only a startup option on the command-line, like `docker run ... %%IMAGE%% --icu-language sv`. +- Use an `arangod.conf` configuration file that sets a language and mount it into the container. For example, create a configuration file on your host system in which you set `icu-language = sv` at the top (before any `[section]`) and then mount the file over the default configuration file like `docker run -v /your-host-path/arangod.conf:/etc/arangodb3/arangod.conf ...`. If you don't specify a language explicitly, the default is `en_US` up to ArangoDB v3.11 and `en_US_POSIX` from ArangoDB v3.12 onward. diff --git a/arangodb/license.md b/arangodb/license.md index 0bd79adbca15..8e8b5b9a6a41 100644 --- a/arangodb/license.md +++ b/arangodb/license.md @@ -1,4 +1,4 @@ -The official Docker images of the ArangoDB Community Edition are governed by the [ArangoDB Community License](https://arangodb.com/community-license/). It limits deployments to a 100 GB on dataset size in production and you cannot use it for any commercial purposes, only internal business purposes. +The official Docker images of the ArangoDB Community Edition are governed by the [ArangoDB Community License](https://arangodb.com/community-license/). It limits deployments to a 100 GB on dataset size in production and you cannot use it for any commercial purposes, only internal business purposes. It does not include the right to distribute, embed within other products, or combine with ArangoDB's Enterprise Edition of the software. The source code of the Community Edition is available under the [Business Source License 1.1 (BUSL-1.1)](https://github.com/arangodb/arangodb/blob/devel/LICENSE). Copying, modification, redistribution, non-commercial use, and commercial use in a non-production context are always allowed. Additionally, you can deploy BUSL-licensed ArangoDB source code for any purpose (including production) as long as you are not creating a commercial derivative work or offering, or are including it in a commercial product, application, or service. On the fourth anniversary of the first publicly available distribution of a specific version, the license changes to the permissive Apache 2.0 open-source license. diff --git a/influxdb/compose.yaml b/influxdb/compose.yaml new file mode 100644 index 000000000000..429c09c87461 --- /dev/null +++ b/influxdb/compose.yaml @@ -0,0 +1,15 @@ +# compose.yaml +services: + influxdb3-core: + container_name: influxdb3-core + image: influxdb:3-core + ports: + - 8181:8181 + command: + - influxdb3 + - serve + - --node-id=node0 + - --object-store=file + - --data-dir=/var/lib/influxdb3 + - --plugin-dir=/var/lib/influxdb3-plugins + diff --git a/influxdb/content.fixed.md b/influxdb/content.fixed.md new file mode 100644 index 000000000000..d366be2bfeb5 --- /dev/null +++ b/influxdb/content.fixed.md @@ -0,0 +1,275 @@ +**This README covers all currently supported versions of InfluxDB:** + +- **InfluxDB 3** (Core and Enterprise) +- **InfluxDB Explorer** +- **InfluxDB v2** +- **InfluxDB v1** + +Scroll to the appropriate section below for Docker setup instructions, configuration flags, and usage examples. + +# InfluxDB Docker Image Overview + +%%LOGO%% + +InfluxDB is the time series database platform designed to collect, store, and process large amounts of timestamped data. InfluxDB supports high write and query workloads and is commonly used for: + +- Monitoring infrastructure and applications +- Collecting data from IoT devices +- Storing log and event data + +InfluxDB offers multiple versions and deployment options to meet diverse technical and operational requirements. + +## InfluxDB 3 + +InfluxDB 3, the latest InfluxDB engine, uses Apache Arrow in-memory processing, the DataFusion SQL engine, and diskless object storage architecture. It delivers sub-10ms query response times, unlimited cardinality, and supports both SQL and InfluxQL. + +InfluxDB 3 offers two editions--both provide SQL and InfluxQL query capabilities, an integrated processing engine for real-time data transformation, and seamless integration with InfluxDB 3 Explorer for data visualization and management: + +- **InfluxDB 3 Core**: A free, open source version of the new engine for local development and prototyping. +- **InfluxDB 3 Enterprise**: A production-grade, scalable time series database that includes support for clustering, security, and enterprise features. +- **InfluxDB Explorer**: A web-based interface for visualizing exploring, and managing time series data stored in InfluxDB 3 databases. + +**License key for Enterprise**: To run InfluxDB 3 Enterprise, you need a license key. Start with a free 30-day trial license by selecting the trial option when you first start the server. For more license options, see the [InfluxDB 3 Enterprise documentation](https://docs.influxdata.com/influxdb3/enterprise/admin/license/). + +## Docker Images + +Docker images are pre-configured templates that allow you to quickly deploy InfluxDB 3 Core or Enterprise in a containerized environment. These images include all necessary dependencies and configurations to run InfluxDB efficiently. + +To pull the latest Docker images, use the following commands: + +- **Core**: Pull the latest InfluxDB 3 Core image for local development and prototyping: + + ```bash + docker pull influxdb:3-core + ``` + +- **Enterprise:**: Pull the latest InfluxDB 3 Enterprise image for production-grade deployments + + ```bash + docker pull influxdb:3-enterprise + ``` + +**Note**: Always pull the latest images to ensure you have the most up-to-date features, security patches, and performance improvements. + +## Parameter glossary + +| Parameter | Description | +|------------------|-----------------------------------------------------------------| +| `--host-id` | Unique identifier for the InfluxDB Core server node. Required. | +| `--node-id` | Unique identifier for the node within an Enterprise cluster. | +| `--object-store` | Backend storage type for metadata and WAL files (e.g., `file`). | +| `--data-dir` | Path to the directory for data storage. | + +# How to use this image + +## Start InfluxDB 3 Core + +Run InfluxDB 3 Core using either Docker Compose or Docker CLI. + +### Docker Compose + +To use Docker Compose with persistent storage, create a `compose.yml` file with the following configuration: + +%%COMPOSE%% + +Start the container by using the following command: + +```bash +docker compose pull && docker compose run influxdb3-core +``` + +Stop the container by using following command: + +```bash +docker container ls --filter "name=influxdb3" +docker kill +``` + +### File system object store with docker + +To start InfluxDB 3 Core with persistent storage and expose the default HTTP port (`8181`), run: + +```bash +docker run -d --name influxdb3-core \ + -p 8181:8181 \ + -v $PWD/influxdb3-data:/var/lib/influxdb3 \ + influxdb:3-core influxdb3 serve \ + --node-id my-influxdb-node \ + --object-store file \ + --data-dir /var/lib/influxdb3 +``` + +This command: + +- Maps container port `8181` (HTTP API) to your host +- Mounts the local `influxdb3-data` directory to persist data +- Configures InfluxDB 3 Core to use a file system object store + +Once the container is running, generate an admin token: + +```bash +docker exec -it influxdb3-core influxdb3 generate token --admin +``` + +Use the token to create a database: + +```bash +docker exec -it influxdb3-core influxdb3 create database my_db --token YOUR_ADMIN_TOKEN +``` + +To check the server health: + +```bash +curl http://localhost:8181/health + --header "Authorization: Bearer YOUR_ADMIN_TOKEN" +``` + +## Start InfluxDB 3 Enterprise + +Use the InfluxDB 3 Enterprise Docker image to run a standalone or clustered instance. This section describes how to mount a file system object store using Docker Compose or Docker CLI. + +### Docker Compose + +To use Docker Compose, open your `compose.yml` file and define a service for InfluxDB 3 Enterprise. + +```yaml +services: + influxdb3-enterprise: + container_name: influxdb3-enterprise + image: influxdb:3-enterprise + ports: + - 8181:8181 + command: + - influxdb3 + - serve + - --node-id=node0 + - --cluster-id=cluster0 + - --object-store=file + - --data-dir=/var/lib/influxdb3 + - --plugin-dir=/var/lib/influxdb3-plugins + environment: + - INFLUXDB3_LICENSE_EMAIL=EMAIL_ADDRESS +``` + +- Replace `EMAIL_ADDRESS` with your email address to bypass the email prompt when generating a trial or at-home license. + +Start your container: + +```bash +docker compose pull && docker compose run influxdb3-enterprise +``` + +- InfluxDB 3 starts in a container with host port 8181 mapped to container port 8181, the server default for HTTP connections. + +To stop your container run: + +```bash +docker container ls --filter "name=influxdb3" +docker kill +``` + +### File system object store with Docker + +To run the Docker image and persist data to the local file system, mount a volume for the object store. + +```bash +docker run -it \ + --volume /path/on/host:/path/in/container \ + influxdb:3-enterprise influxdb3 serve \ + --node-id my_host \ + --cluster-id my_cluster \ + --object-store file \ + --data-dir /path/in/container +``` + +### Configure Docker environment variable + +To defne environment variables in a `.env` file and reference them when starting your InfluxDB 3 Enterprise container. + +Create a `.env` file: + +```bash +INFLUX_LICENSE_KEY=your_license_key_here +INFLUXDB3_LICENSE_EMAIL=you@example.com +``` + +Start the container: + +```bash +docker run -d --name influxdb3-enterprise \ + --env-file .env \ + -v $PWD/data:/var/lib/influxdb3 \ + -p 8181:8181 \ + influxdb:3-enterprise influxdb3 serve \ + --cluster-id cluster1 \ + --node-id node1 \ + --object-store file \ + --data-dir /var/lib/influxdb3 +``` + +This approach lets you: + +- Store sensitive credentials and configuration separately from your shell history or scripts. +- Reuse the same `.env` file in Docker Compose or CI pipelines. + +Generate an admin token: + +```bash +docker exec -it influxdb3-enterprise influxdb3 create token --admin +``` + +Use the token from the output to create a database. + +```bash +docker exec -it influxdb3-enterprise influxdb3 create database enterprise_db --token +``` + +# InfluxDB Explorer + +InfluxDB Explorer provides a graphical interface for visualizing and managing your time series data. Use it alongside your InfluxDB instance to create dashboards, explore metrics, and monitor your systems in real time. + +Access Explorer through the official Docker image: + +```bash +docker pull influxdata/influxdb3-ui:1.0.0 +``` + +For detailed instructions on connecting Explorer to your InfluxDB instance, see [InfluxDB Explorer documentation](https://docs.influxdata.com/influxdb/v3/explorer/). + +# InfluxDB v2 + +## How to use the InfluxDB v2 Docker image + +Use the official [InfluxDB v2 Docker image](https://hub.docker.com/_/influxdb) to start an instance for development or testing. + +```bash +docker run -d -p 8086:8086 \ + -v $PWD/data:/var/lib/influxdb2 \ + -v $PWD/config:/etc/influxdb2 \ + -e DOCKER_INFLUXDB_INIT_MODE=setup \ + -e DOCKER_INFLUXDB_INIT_USERNAME=my-user \ + -e DOCKER_INFLUXDB_INIT_PASSWORD=my-password \ + -e DOCKER_INFLUXDB_INIT_ORG=my-org \ + -e DOCKER_INFLUXDB_INIT_BUCKET=my-bucket \ + influxdb:2 +``` + +After the container starts, visit [http://localhost:8086](http://localhost:8086) in your browser to view the UI. + +For detailed instructions on using Docker Compose with InfluxDB v2, see the [Docker Compose installation guide](https://docs.influxdata.com/influxdb/v2/install/use-docker-compose/). + +# InfluxDB v1 + +## How to use the InfluxDB v1 Docker image + +Use the official [InfluxDB v1 Docker image](https://hub.docker.com/_/influxdb) to start a basic instance for development or testing: + +```bash +docker run -d -p 8086:8086 \ + -v $PWD:/var/lib/influxdb \ + influxdb:1.8 +``` + +This command maps port `8086` and mounts your current directory to persist data. + +For more information, see the [InfluxDB v1 Docker documentation](https://docs.influxdata.com/influxdb/v1/). diff --git a/influxdb/content.md b/influxdb/content.md index e94ab3d60fae..aefbd2b17193 100644 --- a/influxdb/content.md +++ b/influxdb/content.md @@ -1,643 +1,303 @@ -# What is InfluxDB? +**This README covers all currently supported versions of InfluxDB:** -InfluxDB is the time series data platform designed to handle high write and query workloads. Using InfluxDB, you can collect, store, and process large amounts of timestamped data, including metrics and events for use cases such as DevOps monitoring, application metrics, IoT sensors, and event monitoring. +- **InfluxDB 3** (Core and Enterprise) +- **InfluxDB Explorer** +- **InfluxDB v2** +- **InfluxDB v1** -Use the InfluxDB Docker Hub image to write, query, and process time series data in [InfluxDB v2](https://docs.influxdata.com/influxdb/v2/) or [InfluxDB v1](https://docs.influxdata.com/influxdb/v1/). +Scroll to the appropriate section below for Docker setup instructions, configuration flags, and usage examples. -For more information, visit https://influxdata.com. +# InfluxDB Docker Image Overview %%LOGO%% -# How to use this image for InfluxDB v2 +InfluxDB is the time series database platform designed to collect, store, and process large amounts of timestamped data. InfluxDB supports high write and query workloads and is commonly used for: -**Quick start**: See the guide to [Install InfluxDB v2 for Docker](https://docs.influxdata.com/influxdb/v2/install/?t=Docker) and get started using InfluxDB v2. +- Monitoring infrastructure and applications +- Collecting data from IoT devices +- Storing log and event data -## Start InfluxDB v2 and set up with the UI, CLI, or API +InfluxDB offers multiple versions and deployment options to meet diverse technical and operational requirements. -To start an InfluxDB v2 container, enter the following command: +## InfluxDB 3 -```bash -docker run \ - -p 8086:8086 \ - -v "$PWD/data:/var/lib/influxdb2" \ - -v "$PWD/config:/etc/influxdb2" \ - %%IMAGE%%:2 -``` +InfluxDB 3, the latest InfluxDB engine, uses Apache Arrow in-memory processing, the DataFusion SQL engine, and diskless object storage architecture. It delivers sub-10ms query response times, unlimited cardinality, and supports both SQL and InfluxQL. + +InfluxDB 3 offers two editions--both provide SQL and InfluxQL query capabilities, an integrated processing engine for real-time data transformation, and seamless integration with InfluxDB 3 Explorer for data visualization and management: -Replace the following with your own values: +- **InfluxDB 3 Core**: A free, open source version of the new engine for local development and prototyping. +- **InfluxDB 3 Enterprise**: A production-grade, scalable time series database that includes support for clustering, security, and enterprise features. +- **InfluxDB Explorer**: A web-based interface for visualizing exploring, and managing time series data stored in InfluxDB 3 databases. -- `$PWD/data`: A host directory to mount at the container's [InfluxDB data directory](https://docs.influxdata.com/influxdb/v2/reference/internals/file-system-layout/?t=docker#file-system-layout) path -- `$PWD/config`: A host directory to mount at the container's [InfluxDB configuration directory](https://docs.influxdata.com/influxdb/v2/reference/internals/file-system-layout/?t=docker#file-system-layout) path +**License key for Enterprise**: To run InfluxDB 3 Enterprise, you need a license key. Start with a free 30-day trial license by selecting the trial option when you first start the server. For more license options, see the [InfluxDB 3 Enterprise documentation](https://docs.influxdata.com/influxdb3/enterprise/admin/license/). -After the container starts, the InfluxDB UI and API are accessible at http://localhost:8086 on the host. You're ready to set up an initial admin user, token, and bucket from outside or inside the container--choose one of the following: +## Docker Images -- **Set up InfluxDB from outside the container**: [Set up InfluxDB](https://docs.influxdata.com/influxdb/v2/get-started/setup/) from the host or network using the InfluxDB UI, `influx` CLI, or HTTP API. +Docker images are pre-configured templates that allow you to quickly deploy InfluxDB 3 Core or Enterprise in a containerized environment. These images include all necessary dependencies and configurations to run InfluxDB efficiently. -- **Set up InfluxDB from inside the container**: Use `docker exec` to run the `influx` CLI installed in the container--for example: +To pull the latest Docker images, use the following commands: + +- **Core**: Pull the latest InfluxDB 3 Core image for local development and prototyping: ```bash - docker exec influxdb2 influx setup \ - --username $USERNAME \ - --password $PASSWORD \ - --org $ORGANIZATION \ - --bucket $BUCKET \ - --force + docker pull influxdb:3-core ``` -See the [`influx setup` documentation](https://docs.influxdata.com/influxdb/v2/reference/cli/influx/setup/) for the full list of options. +- **Enterprise:**: Pull the latest InfluxDB 3 Enterprise image for production-grade deployments -*If you run setup from within the container, InfluxDB stores `influx` CLI [connection configurations](/influxdb/v2/reference/cli/influx/#provide-required-authentication-credentials) in the container's `/etc/influxdb2/influx-configs` file.* + ```bash + docker pull influxdb:3-enterprise + ``` -## Start InfluxDB v2 with automated setup +**Note**: Always pull the latest images to ensure you have the most up-to-date features, security patches, and performance improvements. -To start and set up InfluxDB v2 with a single command, specify `-e DOCKER_INFLUXDB_INIT_MODE=setup` and `-e DOCKER_INFLUXDB_INIT_` environment variables for the initial user, password, bucket, and organization--for example: +## Parameter glossary -```bash -docker run -d -p 8086:8086 \ - -v "$PWD/data:/var/lib/influxdb2" \ - -v "$PWD/config:/etc/influxdb2" \ - -e DOCKER_INFLUXDB_INIT_MODE=setup \ - -e DOCKER_INFLUXDB_INIT_USERNAME= \ - -e DOCKER_INFLUXDB_INIT_PASSWORD= \ - -e DOCKER_INFLUXDB_INIT_ORG= \ - -e DOCKER_INFLUXDB_INIT_BUCKET= \ - %%IMAGE%%:2 -``` +| Parameter | Description | +|------------------|-----------------------------------------------------------------| +| `--host-id` | Unique identifier for the InfluxDB Core server node. Required. | +| `--node-id` | Unique identifier for the node within an Enterprise cluster. | +| `--object-store` | Backend storage type for metadata and WAL files (e.g., `file`). | +| `--data-dir` | Path to the directory for data storage. | -Replace the following with your own values: +# How to use this image -- `$PWD/data`: A host directory to mount at the container's [InfluxDB data directory](https://docs.influxdata.com/influxdb/v2/reference/internals/file-system-layout/?t=docker#file-system-layout) path -- `$PWD/config`: A host directory to mount at the container's [InfluxDB configuration directory](https://docs.influxdata.com/influxdb/v2/reference/internals/file-system-layout/?t=docker#file-system-layout) path -- ``: A name for your initial admin [user](https://docs.influxdata.com/influxdb/v2/admin/users/) -- ``: A password for your initial admin [user](https://docs.influxdata.com/influxdb/v2/admin/users/) -- ``: A name for your initial [organization](https://docs.influxdata.com/influxdb/v2/admin/organizations/) -- ``: A name for your initial [bucket](https://docs.influxdata.com/influxdb/v2/admin/buckets/) (database) +## Start InfluxDB 3 Core -*If you run setup from within the container, InfluxDB stores `influx` CLI [connection configurations](/influxdb/v2/reference/cli/influx/#provide-required-authentication-credentials) in the container's `/etc/influxdb2/influx-configs` file.* +Run InfluxDB 3 Core using either Docker Compose or Docker CLI. -### Automated setup options +### Docker Compose -In setup mode (`DOCKER_INFLUXDB_INIT_MODE=setup`) or upgrade mode (`DOCKER_INFLUXDB_INIT_MODE=upgrade`), you can specify the following Docker-specific environment variables to provide initial setup values: +To use Docker Compose with persistent storage, create a `compose.yml` file with the following configuration: -- `DOCKER_INFLUXDB_INIT_USERNAME`: A name for your initial admin [user](https://docs.influxdata.com/influxdb/v2/admin/users/). -- `DOCKER_INFLUXDB_INIT_PASSWORD`: A password for your initial admin [user](https://docs.influxdata.com/influxdb/v2/admin/users/). -- `DOCKER_INFLUXDB_INIT_ORG`: A name for your initial [organization](https://docs.influxdata.com/influxdb/v2/admin/organizations/). -- `DOCKER_INFLUXDB_INIT_BUCKET`: A name for your initial [bucket](https://docs.influxdata.com/influxdb/v2/admin/buckets/). -- Optional: `DOCKER_INFLUXDB_INIT_RETENTION`: A [duration](https://docs.influxdata.com/influxdb/v2/reference/glossary/#duration) to use as the initial bucket's [retention period](https://docs.influxdata.com/influxdb/v2/reference/internals/data-retention/#bucket-retention-period). Default: `0` (infinite; doesn't delete data). -- Optional: `DOCKER_INFLUXDB_INIT_ADMIN_TOKEN`: A string value to set for the [Operator token](https://docs.influxdata.com/influxdb/v2/admin/tokens/#operator-token). Default: a generated token. +%%COMPOSE%% -The following example shows how to pass values for all initial setup options: +Start the container by using the following command: ```bash -docker run -d -p 8086:8086 \ - -v "$PWD/data:/var/lib/influxdb2" \ - -v "$PWD/config:/etc/influxdb2" \ - -e DOCKER_INFLUXDB_INIT_MODE=setup \ - -e DOCKER_INFLUXDB_INIT_USERNAME=my-user \ - -e DOCKER_INFLUXDB_INIT_PASSWORD=my-password \ - -e DOCKER_INFLUXDB_INIT_ORG=my-org \ - -e DOCKER_INFLUXDB_INIT_BUCKET=my-bucket \ - -e DOCKER_INFLUXDB_INIT_RETENTION=1w \ - -e DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=my-super-secret-auth-token \ - %%IMAGE%%:2 +docker compose pull && docker compose run influxdb3-core ``` -*To upgrade from InfluxDB 1.x to InfluxDB 2.x, see the **Upgrading from InfluxDB 1.x** section below.\* - -With InfluxDB set up and running, see the [Get started](https://docs.influxdata.com/influxdb/v2/get-started/) tutorial to create tokens and write and query data. - -### Custom Initialization Scripts - -In `setup` mode (`DOCKER_INFLUXDB_INIT_MODE=setup`) or `upgrade` mode (`DOCKER_INFLUXDB_INIT_MODE=upgrade`), the InfluxDB Docker Hub image supports running custom initialization scripts. After the setup process completes, scripts are executed in lexical sort order by name. - -For the container to run scripts, they must: - -- Be mounted in the container's `/docker-entrypoint-initdb.d` directory -- Be named using the `.sh` file name extension -- Be executable by the user running the `docker run` command--for example, to allow the current use to execute a script with `docker run`: - - ```bash - chmod +x ./scripts/ - ``` - -> #### Grant permissions to mounted files -> -> By default, Docker runs containers using the user and group IDs of the user executing the `docker run` command. When files are bind-mounted into the container, Docker preserves the user and group ownership from the host system. - -The image exports a number of variables into the environment before executing scripts. The following variables are available for you to use in your scripts: - -- `INFLUX_CONFIGS_PATH`: Path to the `influx` CLI connection configurations file written by `setup`/`upgrade` -- `INFLUX_HOST`: URL to the `influxd` instance running `setup`/`upgrade` -- `DOCKER_INFLUXDB_INIT_USER_ID`: ID of the initial admin user created by `setup`/`upgrade` -- `DOCKER_INFLUXDB_INIT_ORG_ID`: ID of the initial organization created by `setup`/`upgrade` -- `DOCKER_INFLUXDB_INIT_BUCKET_ID`: ID of the initial bucket created by `setup`/`upgrade` - -For example, to grant an InfluxDB 1.x client *write* permission to your initial bucket, create a `$PWD/scripts/setup-v1.sh` file that contains the following: +Stop the container by using following command: ```bash -#!/bin/bash -set -e - -influx v1 dbrp create \ - --bucket-id ${DOCKER_INFLUXDB_INIT_BUCKET_ID} \ - --db ${V1_DB_NAME} \ - --rp ${V1_RP_NAME} \ - --default \ - --org ${DOCKER_INFLUXDB_INIT_ORG} - -influx v1 auth create \ - --username ${V1_AUTH_USERNAME} \ - --password ${V1_AUTH_PASSWORD} \ - --write-bucket ${DOCKER_INFLUXDB_INIT_BUCKET_ID} \ - --org ${DOCKER_INFLUXDB_INIT_ORG} +docker container ls --filter "name=influxdb3" +docker kill ``` -Then, run the following command to start and set up InfluxDB using custom scripts: +### File system object store with docker + +To start InfluxDB 3 Core with persistent storage and expose the default HTTP port (`8181`), run: ```bash -docker run -p 8086:8086 \ - -v "$PWD/data:/var/lib/influxdb2" \ - -v "$PWD/config:/etc/influxdb2" \ - -v "$PWD/scripts:/docker-entrypoint-initdb.d" \ - -e DOCKER_INFLUXDB_INIT_MODE=setup \ - -e DOCKER_INFLUXDB_INIT_USERNAME=my-user \ - -e DOCKER_INFLUXDB_INIT_PASSWORD=my-password \ - -e DOCKER_INFLUXDB_INIT_ORG=my-org \ - -e DOCKER_INFLUXDB_INIT_BUCKET=my-bucket \ - -e V1_DB_NAME=v1-db \ - -e V1_RP_NAME=v1-rp \ - -e V1_AUTH_USERNAME=v1-user \ - -e V1_AUTH_PASSWORD=v1-password \ - %%IMAGE%%:2 +docker run -d --name influxdb3-core \ + -p 8181:8181 \ + -v $PWD/influxdb3-data:/var/lib/influxdb3 \ + influxdb:3-core influxdb3 serve \ + --node-id my-influxdb-node \ + --object-store file \ + --data-dir /var/lib/influxdb3 ``` -> #### Automated setup and upgrade ignored if already setup -> -> Automated `setup`, `upgrade`, and custom initialization scripts won't run if an existing `influxd.bolt` boltdb file from a previous setup is found in the configured data directory. -> -> This behavior allows for the InfluxDB container to reboot post-setup and avoid overwriting migrated data, `DB is already set up` errors, and errors from non-idempotent script commands. - -## Access InfluxDB v2 file system and ports - -When starting an InfluxDB container, we recommend the following for easy access to your data, configurations, and InfluxDB v2 instance: - -- Publish the container's `8086` port to make the InfluxDB [UI](https://docs.influxdata.com/influxdb/v2/get-started/#influxdb-user-interface-ui) and [HTTP API](https://docs.influxdata.com/influxdb/v2/reference/api/) accessible from the host system. -- Use Docker [Volumes](https://docs.docker.com/storage/volumes/) or [Bind mounts](https://docs.docker.com/storage/bind-mounts/) to persist InfluxDB [data and configuration directories](https://docs.influxdata.com/influxdb/v2/reference/internals/file-system-layout/?t=docker#file-system-layout) outside of containers. - -### Default file system and networking ports - -For InfluxDB v2, the InfluxDB Docker Hub image uses the following default ports and file system paths: - -- TCP port `8086`: the default port for the InfluxDB [UI](https://docs.influxdata.com/influxdb/v2/get-started/#influxdb-user-interface-ui) and [HTTP API](https://docs.influxdata.com/influxdb/v2/reference/api/). To specify a different port or address, use the [`http-bind-address` configuration option](https://docs.influxdata.com/influxdb/v2/reference/config-options/#http-bind-address). -- `/var/lib/influxdb2/`: the [InfluxDB data directory](https://docs.influxdata.com/influxdb/v2/reference/internals/file-system-layout/?t=docker#file-system-layout) - - - `/engine/`: Default InfluxDB [Storage engine path](https://docs.influxdata.com/influxdb/v2/reference/internals/file-system-layout/#engine-path) - - `influxd.bolt`: Default [Bolt path](https://docs.influxdata.com/influxdb/v2/reference/internals/file-system-layout/#bolt-path) - - `influxd.sqlite`: Default [SQLite path](https://docs.influxdata.com/influxdb/v2/reference/internals/file-system-layout/#sqlite-path) - -- `/etc/influxdb2`: the [InfluxDB configuration directory](https://docs.influxdata.com/influxdb/v2/reference/internals/file-system-layout/?t=docker#file-system-layout) - - - `/etc/influxdb2/configs`: `influx` CLI connection configurations file - - `/etc/influxdb2/influx-configs`: `influx` CLI connection configurations file, *if you run setup from within the container* - - Optional: `/etc/influxdb2/config.[yml, json, toml]`: Your customized InfluxDB [configuration options](https://docs.influxdata.com/influxdb/v2/reference/config-options/) file - -### Configure InfluxDB v2 in a container - -To customize InfluxDB, specify [server configuration options](https://docs.influxdata.com/influxdb/v2/reference/config-options/#configuration-options) in a configuration file, environment variables, or command line flags. - -#### Use a configuration file - -To customize and mount an InfluxDB configuration file, do the following: +This command: -1. If you haven't already, [set up InfluxDB](https://docs.influxdata.com/influxdb/v2/get-started/setup/) to initialize an API [Operator token](https://docs.influxdata.com/influxdb/v2/admin/tokens/#operator-token). You'll need the Operator token in the next step. +- Maps container port `8181` (HTTP API) to your host +- Mounts the local `influxdb3-data` directory to persist data +- Configures InfluxDB 3 Core to use a file system object store -2. Run the `influx server-config` CLI command to output the current server configuration to a file in the mounted configuration directory--for example, enter the following command to use the container's `influx` CLI and default Operator token: - - ```bash - docker exec -it influxdb2 influx server-config > "$PWD/config/config.yml" - ``` - -Replace `$PWD/config/` with the host directory that you mounted at the container's `/etc/influxdb2` InfluxDB configuration directory path. - -1. Edit the `config.yml` file to customize [server configuration options](https://docs.influxdata.com/influxdb/v2/reference/config-options/#configuration-options). -2. Restart the container. - - ```bash - docker restart influxdb2 - ``` - -#### Use environment variables and command line flags - -To override specific [configuration options](https://docs.influxdata.com/influxdb/v2/reference/config-options/#configuration-options), use environment variables or command line flags. - -- Pass `INFLUXD_` environment variables to Docker to override the configuration file--for example: - - ```bash - docker run -p 8086:8086 \ - -e INFLUXD_STORAGE_WAL_FSYNC_DELAY=15m \ - influxdb:2 - ``` - -- Pass `influxd` command line flags to override environment variables and the configuration file--for example: - - ```bash - docker run -p 8086:8086 \ - %%IMAGE%%:2 --storage-wal-fsync-delay=15m - ``` - -To learn more, see [InfluxDB configuration options](https://docs.influxdata.com/influxdb/v2/reference/config-options). - -### Upgrading from InfluxDB 1.x - -InfluxDB 2.x provides a [1.x-compatible API](https://docs.influxdata.com/influxdb/v2/reference/api/influxdb-1x/), but expects a different storage layout on disk. To account for these differences, the InfluxDB Docker Hub image provides an `upgrade` feature that migrates 1.x data and configuration to 2.x before starting the `influxd` server. - -The automated upgrade process creates the following in the InfluxDB v2 container: - -- an initial admin user -- an initial organization -- an initial bucket -- InfluxDB v2 data files (the default path is `/var/lib/influxdb2`) -- InfluxDB v2 configuration files (the default path is `/etc/influxdb2`) - -*Mount volumes at both paths to avoid losing data.* - -To run the automated upgrade, specify the following when you start the container: - -- InfluxDB v2 initialization environment variables: - - - `DOCKER_INFLUXDB_INIT_MODE=upgrade` - - `DOCKER_INFLUXDB_INIT_USERNAME`: A name for the initial admin [user](https://docs.influxdata.com/influxdb/v2/admin/users/) - - `DOCKER_INFLUXDB_INIT_PASSWORD`: A password for the initial admin [user](https://docs.influxdata.com/influxdb/v2/admin/users/) - - `DOCKER_INFLUXDB_INIT_ORG`: A name for the initial [organization](https://docs.influxdata.com/influxdb/v2/admin/organizations/) - - `DOCKER_INFLUXDB_INIT_BUCKET`: A name for the initial [bucket](https://docs.influxdata.com/influxdb/v2/admin/buckets/) - - Optional: `DOCKER_INFLUXDB_INIT_RETENTION`: A [duration](https://docs.influxdata.com/influxdb/v2/reference/glossary/#duration) for the bucket [retention period](https://docs.influxdata.com/influxdb/v2/reference/internals/data-retention/#bucket-retention-period). Default: `0` (infinite; doesn't delete data) - - Optional: `DOCKER_INFLUXDB_INIT_ADMIN_TOKEN`: A value to set for the [Operator token](https://docs.influxdata.com/influxdb/v2/admin/tokens/#operator-token). Default: generates a token. - -- 1.x data and configuration paths: - - - A 1.x data volume, specified by the `DOCKER_INFLUXDB_INIT_UPGRADE_V1_DIR` environment variable or mounted at `/var/lib/influxdb` - - Optional: a 1.x custom configuration file, specified by the `DOCKER_INFLUXDB_INIT_UPGRADE_V1_CONFIG` environment variable or mounted at `/etc/influxdb/influxdb.conf` - -The upgrade process searches for mounted 1.x data and configuration paths in the following order of precedence: - -1. A configuration file referenced by the `DOCKER_INFLUXDB_INIT_UPGRADE_V1_CONFIG` environment variable -2. A data directory referenced by the `DOCKER_INFLUXDB_INIT_UPGRADE_V1_DIR` environment variable -3. A configuration file mounted at `/etc/influxdb/influxdb.conf` -4. A data directory mounted at `/var/lib/influxdb` - -> #### Automated setup and upgrade ignored if already setup -> -> Automated `setup`, `upgrade`, and custom initialization scripts won't run if an existing `influxd.bolt` boltdb file from a previous setup is found in the configured data directory. -> -> This behavior allows for the InfluxDB container to reboot post-setup and avoid overwriting migrated data, `DB is already set up` errors, and errors from non-idempotent script commands. - -#### Upgrade InfluxDB 1.x: default data path and configuration - -Assume you've been running a minimal InfluxDB 1.x deployment: +Once the container is running, generate an admin token: ```bash -docker run -p 8086:8086 \ - -v influxdb:/var/lib/influxdb \ - %%IMAGE%%:1.8 +docker exec -it influxdb3-core influxdb3 generate token --admin ``` -To upgrade this deployment to InfluxDB 2.x, stop the running InfluxDB 1.x container, and then run the following command: +Use the token to create a database: ```bash -docker run -p 8086:8086 \ - -v influxdb:/var/lib/influxdb \ - -v influxdb2:/var/lib/influxdb2 \ - -e DOCKER_INFLUXDB_INIT_MODE=upgrade \ - -e DOCKER_INFLUXDB_INIT_USERNAME=my-user \ - -e DOCKER_INFLUXDB_INIT_PASSWORD=my-password \ - -e DOCKER_INFLUXDB_INIT_ORG=my-org \ - -e DOCKER_INFLUXDB_INIT_BUCKET=my-bucket \ - %%IMAGE%%:2 +docker exec -it influxdb3-core influxdb3 create database my_db --token YOUR_ADMIN_TOKEN ``` -#### Upgrade InfluxDB 1.x: custom configuration - -Assume you've been running an InfluxDB 1.x deployment with customized configuration (`/etc/influxdb/influxdb.conf`): +To check the server health: ```bash -docker run -p 8086:8086 \ - -v influxdb:/var/lib/influxdb \ - -v "$PWD/influxdb.conf:/etc/influxdb/influxdb.conf" \ - %%IMAGE%%:1.8 +curl http://localhost:8181/health + --header "Authorization: Bearer YOUR_ADMIN_TOKEN" ``` -To upgrade this deployment to InfluxDB 2.x, stop the running InfluxDB 1.x container, and then run the following command: - -```bash -docker run -p 8086:8086 \ - -v influxdb:/var/lib/influxdb \ - -v influxdb2:/var/lib/influxdb2 \ - -v influxdb2-config:/etc/influxdb2 \ - -v "$PWD/influxdb.conf:/etc/influxdb/influxdb.conf" \ - -e DOCKER_INFLUXDB_INIT_MODE=upgrade \ - -e DOCKER_INFLUXDB_INIT_USERNAME=my-user \ - -e DOCKER_INFLUXDB_INIT_PASSWORD=my-password \ - -e DOCKER_INFLUXDB_INIT_ORG=my-org \ - -e DOCKER_INFLUXDB_INIT_BUCKET=my-bucket \ - %%IMAGE%%:2 +## Start InfluxDB 3 Enterprise + +Use the InfluxDB 3 Enterprise Docker image to run a standalone or clustered instance. This section describes how to mount a file system object store using Docker Compose or Docker CLI. + +### Docker Compose + +To use Docker Compose, open your `compose.yml` file and define a service for InfluxDB 3 Enterprise. + +```yaml +services: + influxdb3-enterprise: + container_name: influxdb3-enterprise + image: influxdb:3-enterprise + ports: + - 8181:8181 + command: + - influxdb3 + - serve + - --node-id=node0 + - --cluster-id=cluster0 + - --object-store=file + - --data-dir=/var/lib/influxdb3 + - --plugin-dir=/var/lib/influxdb3-plugins + environment: + - INFLUXDB3_LICENSE_EMAIL=EMAIL_ADDRESS ``` -#### Upgrade InfluxDB 1.x: custom data and configuration paths +- Replace `EMAIL_ADDRESS` with your email address to bypass the email prompt when generating a trial or at-home license. -Assume you've been running an InfluxDB 1.x deployment with data and configuration mounted at custom paths: +Start your container: ```bash -docker run -p 8086:8086 \ - -v influxdb:/root/influxdb/data \ - -v "$PWD/influxdb.conf:/root/influxdb/influxdb.conf" \ - %%IMAGE%%:1.8 -config /root/influxdb/influxdb.conf +docker compose pull && docker compose run influxdb3-enterprise ``` -Before you upgrade to InfluxDB v2, decide whether to keep using your custom paths or to use the InfluxDB v2 defaults. +- InfluxDB 3 starts in a container with host port 8181 mapped to container port 8181, the server default for HTTP connections. -To use InfluxDB v2 defaults, stop the running InfluxDB 1.x container, and then run the following command: +To stop your container run: ```bash -docker run -p 8086:8086 \ - -v influxdb:/root/influxdb/data \ - -v influxdb2:/var/lib/influxdb2 \ - -v influxdb2-config:/etc/influxdb2 \ - -v "$PWD/influxdb.conf:/root/influxdb/influxdb.conf" \ - -e DOCKER_INFLUXDB_INIT_MODE=upgrade \ - -e DOCKER_INFLUXDB_INIT_USERNAME=my-user \ - -e DOCKER_INFLUXDB_INIT_PASSWORD=my-password \ - -e DOCKER_INFLUXDB_INIT_ORG=my-org \ - -e DOCKER_INFLUXDB_INIT_BUCKET=my-bucket \ - -e DOCKER_INFLUXDB_INIT_UPGRADE_V1_CONFIG=/root/influxdb/influxdb.conf \ - %%IMAGE%%:2 +docker container ls --filter "name=influxdb3" +docker kill ``` -To use your custom paths instead of InfluxDB v2 default paths, run the following command: +### File system object store with Docker + +To run the Docker image and persist data to the local file system, mount a volume for the object store. ```bash -docker run -p 8086:8086 \ - -v influxdb:/root/influxdb/data \ - -v influxdb2:/root/influxdb2/data \ - -v influxdb2-config:/etc/influxdb2 \ - -v "$PWD/influxdb.conf:/root/influxdb/influxdb.conf" \ - -e DOCKER_INFLUXDB_INIT_MODE=upgrade \ - -e DOCKER_INFLUXDB_INIT_USERNAME=my-user \ - -e DOCKER_INFLUXDB_INIT_PASSWORD=my-password \ - -e DOCKER_INFLUXDB_INIT_ORG=my-org \ - -e DOCKER_INFLUXDB_INIT_BUCKET=my-bucket \ - -e DOCKER_INFLUXDB_INIT_UPGRADE_V1_CONFIG=/root/influxdb/influxdb.conf \ - -e DOCKER_INFLUXDB_CONFIG_PATH=/root/influxdb2/config.toml \ - -e DOCKER_INFLUXDB_BOLT_PATH=/root/influxdb2/influxdb.bolt \ - -e DOCKER_INFLUXDB_ENGINE_PATH=/root/influxdb2/engine \ - %%IMAGE%%:2 +docker run -it \ + --volume /path/on/host:/path/in/container \ + influxdb:3-enterprise influxdb3 serve \ + --node-id my_host \ + --cluster-id my_cluster \ + --object-store file \ + --data-dir /path/in/container ``` -To learn more about the upgrade process, see the [v1-to-v2 upgrade guide](https://docs.influxdata.com/influxdb/v2.0/upgrade/v1-to-v2/). - -### Upgrading from quay.io-hosted InfluxDB 2.x image - -Early Docker builds of InfluxDB 2.x were hosted at `quay.io/influxdb/influxdb` and contained the `influx` and `influxd` binaries without any default configuration or helper scripts. By default, the `influxd` process stored data in `/root/.influxdbv2`. - -Starting with `v2.0.4`, we restored the InfluxDB Docker Hub build, which defaults to storing data in `/var/lib/influxdb2`. If you upgrade directly from `quay.io/influxdb/influxdb` to `influxdb:2.0.4` using the default settings, InfluxDB won't be able to find your existing data files. +### Configure Docker environment variable -To avoid this problem when migrating from `quay.io/influxdb/influxdb` to `influxdb:2.0`, choose one of the following: +To defne environment variables in a `.env` file and reference them when starting your InfluxDB 3 Enterprise container. -#### Update the mount to use the InfluxDB default - -To use the InfluxDB Docker Hub data path, start a container that mounts your data volume into `/var/lib/influxdb2`--for example, if you used the following command to start the InfluxDB quay.io container: +Create a `.env` file: ```bash -# quay.io InfluxDB 2.x container -docker run -p 8086:8086 \ - -v "$PWD:/root/.influxdbv2" \ - quay.io/influxdb/influxdb:v2.0.3 +INFLUX_LICENSE_KEY=your_license_key_here +INFLUXDB3_LICENSE_EMAIL=you@example.com ``` -Use this command to start an InfluxDB v2 Docker Hub container: +Start the container: ```bash -# Docker Hub InfluxDB 2.x container -docker run -p 8086:8086 \ - -v "$PWD:/var/lib/influxdb2" \ - %%IMAGE%%:2 +docker run -d --name influxdb3-enterprise \ + --env-file .env \ + -v $PWD/data:/var/lib/influxdb3 \ + -p 8181:8181 \ + influxdb:3-enterprise influxdb3 serve \ + --cluster-id cluster1 \ + --node-id node1 \ + --object-store file \ + --data-dir /var/lib/influxdb3 ``` -#### Configure InfluxDB to use the container home directory +This approach lets you: -To continue using the `/root/.influxdbv2` data path, customize storage path configuration options ([bolt-path](https://docs.influxdata.com/influxdb/v2/reference/config-options/#bolt-path), [engine-path](https://docs.influxdata.com/influxdb/v2/reference/config-options/#engine-path), [sqlite-path](https://docs.influxdata.com/influxdb/v2/reference/config-options/#sqlite-path)) configuration options for your InfluxDB Docker Hub container--for example, if you used the following command to start the InfluxDB quay.io container: - -```bash -# quay.io-hosted InfluxDB 2.x -docker run -p 8086:8086 \ - -v "$PWD:/root/.influxdbv2" \ - quay.io/influxdb/influxdb:v2.0.3 -``` +- Store sensitive credentials and configuration separately from your shell history or scripts. +- Reuse the same `.env` file in Docker Compose or CI pipelines. -Use this command to start an InfluxDB v2 Docker Hub container: +Generate an admin token: ```bash -docker run -p 8086:8086 \ - -e INFLUXD_BOLT_PATH=/root/.influxdbv2/influxd.bolt \ - -e INFLUXD_ENGINE_PATH=/root/.influxdbv2/engine \ - -v "$PWD:/root/.influxdbv2" \ - %%IMAGE%%:2 +docker exec -it influxdb3-enterprise influxdb3 create token --admin ``` -# How to use this image for InfluxDB v1 - -Use the InfluxDB Docker Hub image to run and set up an [InfluxDB 1.x](https://docs.influxdata.com/influxdb/v1/) container. - -## Running the container - -To start an InfluxDB 1.x container, enter the following command: +Use the token from the output to create a database. ```bash -docker run -p 8086:8086 \ - -v "$PWD:/var/lib/influxdb" \ - %%IMAGE%%:1.8 +docker exec -it influxdb3-enterprise influxdb3 create database enterprise_db --token ``` -The command passes the following arguments: - -- `-p 8086:8086`: Exposes the InfluxDB [HTTP API](https://docs.influxdata.com/influxdb/v2/reference/api/) on host port `8086`. -- `-v $PWD:/var/lib/influxdb`: Mounts the host's `$PWD` directory to the [InfluxDB data directory](https://docs.influxdata.com/influxdb/v1/concepts/file-system-layout/) to persist data outside the container. - -Replace `$PWD` with the host directory where you want InfluxDB to store data. - -*Use Docker [Volumes](https://docs.docker.com/storage/volumes/) or [Bind mounts](https://docs.docker.com/storage/bind-mounts/) to persist InfluxDB [data and configuration directories](https://docs.influxdata.com/influxdb/v1/concepts/file-system-layout/).* - -## Networking ports - -InfluxDB uses the following networking ports: +# InfluxDB Explorer -- TCP port `8086`: the default port for the [HTTP API](https://docs.influxdata.com/influxdb/v1/tools/api/) -- TCP port `2003`: the port for the Graphite protocol (if enabled) +InfluxDB Explorer provides a graphical interface for visualizing and managing your time series data. Use it alongside your InfluxDB instance to create dashboards, explore metrics, and monitor your systems in real time. -Using the `docker run` [`-P, --publish-all` flag](https://docs.docker.com/reference/cli/docker/container/run/#publish-all) exposes the InfluxDB HTTP API to the host. - -## Configure InfluxDB v1 in a container - -To configure InfluxDB v1 in a container, use a configuration file or environment variables. - -### Use a configuration file - -To customize and mount a configuration file, do the following: - -1. Output the current server configuration to a file in the mounted configuration directory--for example: - - ```bash - docker run --rm %%IMAGE%%:1.8 influxd config > influxdb.conf - ``` - -2. Edit the `influxdb.conf` file to customize [server configuration options](https://docs.influxdata.com/influxdb/v2/reference/config-options/#configuration-options). - - ```bash - docker run -p 8086:8086 \ - -v "$PWD/influxdb.conf:/etc/influxdb/influxdb.conf:ro" \ - %%IMAGE%%:1.8 -config /etc/influxdb/influxdb.conf - ``` - - Replace `$PWD` with the host directory where you want to store the configuration file. - -### Use environment variables - -Pass [`INFLUXDB_` environment variables](https://docs.influxdata.com/influxdb/v1/administration/config/#environment-variables) to override specific InfluxDB v1 configuration options. An environment variable overrides the equivalent option in the configuration file. +Access Explorer through the official Docker image: ```bash -docker run -p 8086:8086 \ - -e INFLUXDB_REPORTING_DISABLED=true \ - -e INFLUXDB_META_DIR=/path/to/metadir \ - -e INFLUXDB_DATA_QUERY_LOG_ENABLED=false \ - %%IMAGE%%:1.8 +docker pull influxdata/influxdb3-ui:1.0.0 ``` -Learn more about [configuring InfluxDB v1](https://docs.influxdata.com/influxdb/v1.8/administration/config/). - -## Graphite +For detailed instructions on connecting Explorer to your InfluxDB instance, see [InfluxDB Explorer documentation](https://docs.influxdata.com/influxdb/v3/explorer/). -InfluxDB supports the Graphite line protocol, but the service and ports are not exposed by default. To run InfluxDB with Graphite support enabled, you can either use a configuration file or set the appropriate environment variables. Run InfluxDB with the default Graphite configuration: - -```bash -docker run -p 8086:8086 -p 2003:2003 \ - -e INFLUXDB_GRAPHITE_ENABLED=true \ - %%IMAGE%%:1.8 -``` +# InfluxDB v2 -See the [README on GitHub](https://github.com/influxdata/influxdb/blob/master/services/graphite/README.md) for more detailed documentation to set up the Graphite service. In order to take advantage of graphite templates, you should use a configuration file by outputting a default configuration file using the steps above and modifying the `[[graphite]]` section. +Use the official [InfluxDB v2 Docker image](https://hub.docker.com/_/influxdb) to start a containerized instance of InfluxDB v2. -## InfluxDB v1 HTTP API +## Start InfluxDB v2 -Creating a DB named mydb: +To start InfluxDB v2 in a container with mounted volumes for persistent configuration and data, run: ```bash -curl -G http://localhost:8086/query --data-urlencode "q=CREATE DATABASE mydb" +docker run -d -p 8086:8086 \ + -v $PWD/data:/var/lib/influxdb2 \ + -v $PWD/config:/etc/influxdb2 \ + -e DOCKER_INFLUXDB_INIT_MODE=setup \ + -e DOCKER_INFLUXDB_INIT_USERNAME=my-user \ + -e DOCKER_INFLUXDB_INIT_PASSWORD=my-password \ + -e DOCKER_INFLUXDB_INIT_ORG=my-org \ + -e DOCKER_INFLUXDB_INIT_BUCKET=my-bucket \ + influxdb:2 ``` -Inserting into the DB: +This command: -```bash -curl -i -XPOST 'http://localhost:8086/write?db=mydb' --data-binary 'cpu_load_short,host=server01,region=us-west value=0.64 1434055562000000000' -``` +- Starts a detached container named `influxdb2` +- Mounts volumes for persistent storage of data and configuration +- Initializes InfluxDB with a user, organization, and bucket -Read more about this in the [official documentation](https://docs.influxdata.com/influxdb/latest/guides/writing_data/). +To access the InfluxDB UI, visit [http://localhost:8086](http://localhost:8086) in your browser. -## CLI / SHELL +## Start an existing InfluxDB v2 container -Start the container: +If you previously created the container and it has been stopped, restart it with: ```bash -docker run --name=influxdb -d -p 8086:8086 %%IMAGE%%:1.8 +docker start influxdb2 ``` -Run the influx client in this container: +Replace `influxdb2` with the name of your container if different. -```bash -docker exec -it influxdb influx -``` +## Stop InfluxDB v2 -Or run the influx client in a separate container: +To stop a running container: ```bash -docker run --rm --link=influxdb -it %%IMAGE%%:1.8 influx -host influxdb +docker stop influxdb2 ``` -## InfluxDB v1 database initialization - -### Not recommended for production - -We **don't** recommend using initialization options for InfluxDB v1 production scenarios, but they're useful when running standalone instances for testing. - -The InfluxDB Docker Hub image lets you set initialization options when creating an InfluxDB v1 container. - -The database initialization script is only called when running `influxd`; it isn't executed by any other program. - -### Environment variables - -During the InfluxDB v1 set up process, the InfluxDB image uses environment variables to automatically configure some server options. You can override the following environment variables to customize set up options. - -#### INFLUXDB_DB - -Automatically initializes a database with the name of this environment variable. - -#### INFLUXDB_HTTP_AUTH_ENABLED - -Enables authentication. Either this must be set or `auth-enabled = true` must be set within the configuration file for any authentication-related options below to work. +This gracefully shuts down the container. -#### INFLUXDB_ADMIN_USER +For detailed instructions on using Docker Compose with InfluxDB v2, see the [Docker Compose installation guide](https://docs.influxdata.com/influxdb/v2/install/use-docker-compose/). -The name of the admin user to be created. If this is unset, no admin user is created. +# InfluxDB v1 -#### INFLUXDB_ADMIN_PASSWORD +## How to use the InfluxDB v1 Docker image -The password for the admin user configured with `INFLUXDB_ADMIN_USER`. If this is unset, a random password is generated and printed to standard out. - -#### INFLUXDB_USER - -The name of a user to be created with no privileges. If `INFLUXDB_DB` is set, this user will be granted read and write permissions for that database. - -#### INFLUXDB_USER_PASSWORD - -The password for the user configured with `INFLUXDB_USER`. If this is unset, a random password is generated and printed to standard out. - -#### INFLUXDB_READ_USER - -The name of a user to be created with read privileges on `INFLUXDB_DB`. If `INFLUXDB_DB` is not set, this user will have no granted permissions. - -#### INFLUXDB_READ_USER_PASSWORD - -The password for the user configured with `INFLUXDB_READ_USER`. If this is unset, a random password is generated and printed to standard out. - -#### INFLUXDB_WRITE_USER - -The name of a user to be created with write privileges on `INFLUXDB_DB`. If `INFLUXDB_DB` is not set, this user will have no granted permissions. - -#### INFLUXDB_WRITE_USER_PASSWORD - -The password for the user configured with `INFLUXDB_WRITE_USER`. If this is unset, a random password is generated and printed to standard out. - -### Initialization Files - -If the Docker image finds any files with the extensions `.sh` or `.iql` inside of the `/docker-entrypoint-initdb.d` folder, it will execute them. The order they are executed in is determined by the shell. This is usually alphabetical order. - -### Manually Initialize InfluxDB v1 - -To manually initialize an InfluxDB v1 database, use `docker run` to call the `/init-influxdb.sh` script directly. The script takes the same initialization options as the `influxd run` command--for example: +Use the official [InfluxDB v1 Docker image](https://hub.docker.com/_/influxdb) to start a basic instance for development or testing: ```bash -docker run --rm \ - -e INFLUXDB_DB=db0 \ - -e INFLUXDB_ADMIN_USER=admin \ - -e INFLUXDB_ADMIN_PASSWORD=supersecretpassword \ - -e INFLUXDB_USER=telegraf -e \ - -e INFLUXDB_USER_PASSWORD=secretpassword \ - -v "$PWD:/var/lib/influxdb" \ - %%IMAGE%%:1.8 /init-influxdb.sh +docker run -d -p 8086:8086 \ + -v $PWD:/var/lib/influxdb \ + influxdb:1.8 ``` -The command creates the following: - -- a database named `db0` -- an admin user `admin` with the password `supersecretpassword` -- a `telegraf` user with the password `secretpassword` +This command maps port `8086` and mounts your current directory to persist data. -The `--rm` flag causes Docker to exit and delete the container after the script runs. The data and configuration files created during initialization remain in the mounted volume (the host's `$PWD` directory). +For more information, see the [InfluxDB v1 Docker documentation](https://docs.influxdata.com/influxdb/v1/). diff --git a/mysql/content.md b/mysql/content.md index 0e24f5399fd6..9b21973bd9ca 100644 --- a/mysql/content.md +++ b/mysql/content.md @@ -56,7 +56,13 @@ $ docker logs some-%%REPO%% ## Using a custom MySQL configuration file -The default configuration for MySQL can be found in `/etc/mysql/my.cnf`, which may `!includedir` additional directories such as `/etc/mysql/conf.d` or `/etc/mysql/mysql.conf.d`. Please inspect the relevant files and directories within the `%%IMAGE%%` image itself for more details. +The default configuration for MySQL varies depending on the base image: + +**Oracle-based images (default):** The default configuration is located at `/etc/my.cnf`, which may `!includedir` additional directories such as `/etc/mysql/conf.d`. + +**Debian-based MySQL 8 images:** The default configuration can be found in `/etc/mysql/my.cnf`, which may `!includedir` additional directories such as `/etc/mysql/conf.d`. + +Please inspect the relevant files and directories within the `%%IMAGE%%` image itself for more details. If `/my/custom/config-file.cnf` is the path and name of your custom configuration file, you can start your `%%IMAGE%%` container like this (note that only the directory path of the custom config file is used in this command): @@ -64,7 +70,7 @@ If `/my/custom/config-file.cnf` is the path and name of your custom configuratio $ docker run --name some-%%REPO%% -v /my/custom:/etc/mysql/conf.d -e MYSQL_ROOT_PASSWORD=my-secret-pw -d %%IMAGE%%:tag ``` -This will start a new container `some-%%REPO%%` where the MySQL instance uses the combined startup settings from `/etc/mysql/my.cnf` and `/etc/mysql/conf.d/config-file.cnf`, with settings from the latter taking precedence. +This will start a new container `some-%%REPO%%` where the MySQL instance uses the combined startup settings from the default configuration file and `/etc/mysql/conf.d/config-file.cnf`, with settings from the latter taking precedence. ### Configuration without a `cnf` file diff --git a/ros/content.md b/ros/content.md index 86033fbbfda8..4544e8ab5141 100644 --- a/ros/content.md +++ b/ros/content.md @@ -1,10 +1,10 @@ -# What is [ROS](https://www.ros.org/)? +# What is [ROS](https://docs.ros.org/)? The Robot Operating System (ROS) is a set of software libraries and tools that help you build robot applications. From drivers to state-of-the-art algorithms, and with powerful developer tools, ROS has what you need for your next robotics project. And it's all open source. > [wikipedia.org/wiki/Robot_Operating_System](https://en.wikipedia.org/wiki/Robot_Operating_System) -[%%LOGO%%](https://www.ros.org/) +[%%LOGO%%](https://docs.ros.org/) # How to use this image @@ -13,7 +13,7 @@ The Robot Operating System (ROS) is a set of software libraries and tools that h To create your own ROS docker images and install custom packages, here's a simple example of installing the C++, Python client library demos using the official released Debian packages via apt-get. ```dockerfile -FROM %%IMAGE%%:foxy +FROM %%IMAGE%%:rolling-ros-core as aptgetter # install ros package RUN apt-get update && apt-get install -y \ @@ -28,8 +28,8 @@ CMD ["ros2", "launch", "demo_nodes_cpp", "talker_listener_launch.py"] Note: all ROS images include a default entrypoint that sources the ROS environment setup before executing the configured command, in this case the demo packages launch file. You can then build and run the Docker image like so: ```console -$ docker build -t my/ros:app . -$ docker run -it --rm my/ros:app +$ docker build -t my/ros:aptgetter . +$ docker run -it --rm my/ros:aptgetter [INFO] [launch]: process[talker-1]: started with pid [813] [INFO] [launch]: process[listener-2]: started with pid [814] [INFO] [talker]: Publishing: 'Hello World: 1' @@ -44,59 +44,85 @@ $ docker run -it --rm my/ros:app To create your own ROS docker images and build custom packages, here's a simple example of installing a package's build dependencies, compiling it from source, and installing the resulting build artifacts into a final multi-stage image layer. ```dockerfile -ARG FROM_IMAGE=%%IMAGE%%:foxy +ARG FROM_IMAGE=%%IMAGE%%:rolling ARG OVERLAY_WS=/opt/ros/overlay_ws # multi-stage for caching FROM $FROM_IMAGE AS cacher +ARG OVERLAY_WS + +# overwrite defaults to persist minimal cache +RUN rosdep update --rosdistro $ROS_DISTRO && \ + cat < /etc/apt/apt.conf.d/docker-clean && apt-get update +APT::Install-Recommends "false"; +APT::Install-Suggests "false"; +EOF # clone overlay source -ARG OVERLAY_WS WORKDIR $OVERLAY_WS/src -RUN echo "\ -repositories: \n\ - ros2/demos: \n\ - type: git \n\ - url: https://github.com/ros2/demos.git \n\ - version: ${ROS_DISTRO} \n\ -" > ../overlay.repos -RUN vcs import ./ < ../overlay.repos - -# copy manifests for caching -WORKDIR /opt -RUN mkdir -p /tmp/opt && \ - find ./ -name "package.xml" | \ - xargs cp --parents -t /tmp/opt && \ - find ./ -name "COLCON_IGNORE" | \ - xargs cp --parents -t /tmp/opt || true +RUN cat < /tmp/${type}_debs.txt +done +EOF # multi-stage for building FROM $FROM_IMAGE AS builder - -# install overlay dependencies ARG OVERLAY_WS -WORKDIR $OVERLAY_WS -COPY --from=cacher /tmp/$OVERLAY_WS/src ./src -RUN . /opt/ros/$ROS_DISTRO/setup.sh && \ - apt-get update && rosdep install -y \ - --from-paths \ - src/ros2/demos/demo_nodes_cpp \ - src/ros2/demos/demo_nodes_py \ - --ignore-src \ - && rm -rf /var/lib/apt/lists/* + +# install build dependencies +COPY --from=cacher /tmp/build_debs.txt /tmp/build_debs.txt +RUN --mount=type=cache,target=/etc/apt/apt.conf.d,from=cacher,source=/etc/apt/apt.conf.d \ + --mount=type=cache,target=/var/lib/apt/lists,from=cacher,source=/var/lib/apt/lists \ + --mount=type=cache,target=/var/cache/apt,sharing=locked \ + < /tmp/build_debs.txt xargs apt-get install -y # build overlay source +WORKDIR $OVERLAY_WS COPY --from=cacher $OVERLAY_WS/src ./src -ARG OVERLAY_MIXINS="release" RUN . /opt/ros/$ROS_DISTRO/setup.sh && \ colcon build \ --packages-select \ demo_nodes_cpp \ demo_nodes_py \ - --mixin $OVERLAY_MIXINS + --mixin release + +# multi-stage for running +FROM $FROM_IMAGE-ros-core AS runner +ARG OVERLAY_WS + +# install exec dependencies +COPY --from=cacher /tmp/exec_debs.txt /tmp/exec_debs.txt +RUN --mount=type=cache,target=/etc/apt/apt.conf.d,from=cacher,source=/etc/apt/apt.conf.d \ + --mount=type=cache,target=/var/lib/apt/lists,from=cacher,source=/var/lib/apt/lists \ + --mount=type=cache,target=/var/cache/apt,sharing=locked \ + < /tmp/exec_debs.txt xargs apt-get install -y -# source entrypoint setup -ENV OVERLAY_WS $OVERLAY_WS +# setup overlay install +ENV OVERLAY_WS=$OVERLAY_WS +COPY --from=builder $OVERLAY_WS/install $OVERLAY_WS/install RUN sed --in-place --expression \ '$isource "$OVERLAY_WS/install/setup.bash"' \ /ros_entrypoint.sh @@ -105,11 +131,36 @@ RUN sed --in-place --expression \ CMD ["ros2", "launch", "demo_nodes_cpp", "talker_listener_launch.py"] ``` -The example above starts by using [`vcstool`](https://github.com/dirk-thomas/vcstool) to clone source repos of interest into the cacher stage. One could similarly `COPY` code from the local build context into the source directory as well. Package manifest files are then cached in a temporary directory where the following builder stage may copy from to install necessary dependencies with [`rosdep`](https://github.com/ros-infrastructure/rosdep). This is done prior to copying the rest of the source files to preserve the multi-stage build cache, given unaltered manifests do not alter declared dependencies, saving time and bandwidth. The overlay is then built using [`colcon`](https://colcon.readthedocs.io/en/released/), the entrypoint updated to source the workspace, and the default command set to launch the demo. +The example above consists of three sequential stages. The `cacher` stage first updates the apt lists and ROS index, uses [`vcstool`](https://github.com/dirk-thomas/vcstool) to clone a demo repo into the workspace source directory, and derives build and runtime dependency sets using [`rosdep`](https://docs.ros.org/en/rolling/Tutorials/Intermediate/Rosdep.html). The `builder` stage installs the derived build dependencies, sources the ROS install underlay, and compiles the source in release mode using [`colcon`](https://docs.ros.org/en/rolling/Tutorials/Beginner-Client-Libraries/Colcon-Tutorial.html). Finally, the `runner` stage installs only runtime dependencies, copies the compiled workspace artifacts, and sets up the environment to launch the demo. Note the example consists of several subtle optimizations: + +- Multi-Stage Build + - Dependency derivation, compilation, and runtime setup are partitioned + - Maximizes cache retention despite package source or build/runtime changes + - Greater concurrency, e.g., colcon build while runtime apt installs +- Persistent Cache Propagation + - Use of [`--mount`](https://docs.docker.com/engine/reference/builder/#run---mount) to cache temp data without bloating layers + - Maintain temporally consistent apt lists between parallel stages + - Avoid needless network I/O between stages or across Docker rebuilds +- Minimal Image Size + - Final stage builds from `ros-core` for smallest runtime image + - Builds and installs only a select few packages in the workspace + - Only workspace install artifacts are copied into final layers -Note: `--from-paths` and `--packages-select` are set here as so to only install the dependencies and build for the demo C++ and Python packages, among many in the demo git repo that was cloned. To install the dependencies and build all the packages in the source workspace, merely change the scope by setting `--from-paths src/` and dropping the `--packages-select` arguments. +For comparison, the resulting `runner` image is similar in size to the earlier `aptgetter` example. This allows you to develop and distribute custom ROS packages without significantly increasing image size compared to pre-built Debian installations: -For more advance examples such as daisy chaining multiple overlay workspaces to improve caching of docker image build layers, using tools such as ccache to accelerate compilation with colcon, or using buildkit to save build time and bandwidth even when dependencies change, the project `Dockerfile`s in the ROS 2 [Navigation2](https://github.com/ros-planning/navigation2) repo are excellent resources. +```console +$ docker image ls my/ros --format "table {{.Tag}}\t{{.Size}}" +TAG SIZE +aptgetter 504MB +runner 510MB +builder 941MB +$ docker image ls ros --format "table {{.Tag}}\t{{.Size}}" +TAG SIZE +rolling-ros-core 489MB +rolling 876MB +``` + +For more advance examples such as daisy chaining multiple overlay workspaces to improve caching of docker image build layers, using tools such as ccache to accelerate compilation with colcon, or using buildkit to save build time and bandwidth even when dependencies change, the project `Dockerfile`s in the [Navigation2](https://github.com/ros-planning/navigation2) repo are excellent resources. ## Deployment use cases @@ -119,7 +170,7 @@ Developing such complex systems with cutting edge implementations of newly publi With the advancements and standardization of software containers, roboticists are primed to acquire a host of improved developer tooling for building and shipping software. To help alleviate the growing pains and technical challenges of adopting new practices, we have focused on providing an official resource for using ROS with these new technologies. -For a complete listing of supported architectures and base images for each ROS Distribution Release, please read the official REP on target platforms for either [ROS 1](https://www.ros.org/reps/rep-0003.html) or for [ROS 2](https://www.ros.org/reps/rep-2000.html). +For a complete listing of supported architectures and base images for each ROS Distribution Release, please read the official REP on target platforms [here](https://www.ros.org/reps/rep-2001.html). ## Deployment suggestions @@ -127,11 +178,10 @@ The available tags include supported distros along with a hierarchy tags based o - `ros-core`: minimal ROS install - `ros-base`: basic tools and libraries (also tagged with distro name with LTS version as `latest`) -- `ros1-bridge`: tools and libraries to run hybrid ROS 1 - ROS 2 systems and bridge messages between them In the interest of keeping `ros-core` tag minimal in image size, developer tools such as `rosdep`, `colcon` and `vcstools` are not shipped in `ros_core`, but in `ros-base` instead. -The rest of the common meta-packages such as `desktop` are hosted on repos under OSRF's Docker Hub profile [here](https://hub.docker.com/r/osrf/ros/). These meta-packages include graphical dependencies and hook a host of other large packages such as X11, X server, etc. So in the interest of keeping the official images lean and secure, the desktop packages are just being hosted with OSRF's profile. For an extensive list of available variants, please read the official REP on target platforms for either [ROS 1](https://ros.org/reps/rep-0150.html) or for [ROS 2](https://www.ros.org/reps/rep-2001.html). +The rest of the common meta-packages such as `desktop` are hosted on repos under OSRF's Docker Hub profile [here](https://hub.docker.com/r/osrf/ros/). These meta-packages include graphical dependencies and hook a host of other large packages such as X11, X server, etc. So in the interest of keeping the official images lean and secure, the desktop packages are just being hosted with OSRF's profile. ### Volumes @@ -201,59 +251,10 @@ $ docker compose rm > Note: the auto-generated network, `ros_demos_default`, will persist until you explicitly remove it using `docker compose down`. -### ROS 1 Bridge - -To ease ROS 2 migration, [`ros1_bridge`](https://index.ros.org/p/ros1_bridge) is a ROS 2 package that provides bidirectional communication between ROS 1 and ROS 2. As a minimal example, given the ROS 2 Dockerfile above, we'll create the ROS 1 equivalent below, and name the Dockerfile appropriately. - -```dockerfile -FROM %%IMAGE%%:noetic - -# install ros package -RUN apt-get update && apt-get install -y \ - ros-${ROS_DISTRO}-ros-tutorials \ - ros-${ROS_DISTRO}-common-tutorials && \ - rm -rf /var/lib/apt/lists/* - -# launch ros package -CMD ["roslaunch", "roscpp_tutorials", "talker_listener_launch"] -``` - -The compose file bellow spawns services for both talker listener demos while connecting the two via a dynamic bridge. You may then view the log output from both pairs of talker and listener nodes cross talking over the `/chatter` topic. - -```yaml -services: - ros1: - build: - context: ./ - dockerfile: ros1.Dockerfile - - ros2: - build: - context: ./ - dockerfile: ros2.Dockerfile - - bridge: - image: ros:foxy-ros1-bridge - environment: - - "ROS_HOSTNAME=bridge" - - "ROS_MASTER_URI=http://ros1:11311" - command: ros2 run ros1_bridge dynamic_bridge -``` - # More Resources -[ROS.org](http://www.ros.org/): Main ROS website -[Q&A](https://answers.ros.org/questions/): Ask questions. Get answers +[Docs](https://docs.ros.org/): ROS Developer Documentation +[Q&A](https://robotics.stackexchange.com/): Ask questions. Get answers [Forums](https://discourse.ros.org/): Hear the latest discussions -[Blog](http://www.ros.org/news/): Stay up-to-date [Packages](https://index.ros.org/?search_packages=true): Discover indexed packages -[OSRF](https://www.osrfoundation.org/): Open Source Robotics Foundation - -## ROS 2 - -[Index](https://docs.ros.org): ROS 2 Documentation -[Design](https://design.ros2.org/): ROS 2 Design Articles - -## ROS 1 - -[Wiki](http://wiki.ros.org/Documentation): ROS 1 Documentation +[OSRF](https://www.openrobotics.org/): Open Source Robotics Foundation diff --git a/ros/metadata.json b/ros/metadata.json index df07586b5b35..de234f00b76c 100644 --- a/ros/metadata.json +++ b/ros/metadata.json @@ -1,6 +1,8 @@ { "hub": { "categories": [ + "languages-and-frameworks", + "machine-learning-and-ai", "operating-systems" ] }