Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pass file location to lifecycle commands #173

Merged
merged 4 commits into from
Dec 30, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ It handles __recurring or one-off backups of Docker volumes__ to a __local direc
- [Run multiple backup schedules in the same container](#run-multiple-backup-schedules-in-the-same-container)
- [Define different retention schedules](#define-different-retention-schedules)
- [Use special characters in notification URLs](#use-special-characters-in-notification-urls)
- [Handle file uploads using third party tools](#handle-file-uploads-using-third-party-tools)
- [Recipes](#recipes)
- [Backing up to AWS S3](#backing-up-to-aws-s3)
- [Backing up to Filebase](#backing-up-to-filebase)
Expand Down Expand Up @@ -894,6 +895,45 @@ where service is any of the [supported services][shoutrrr-docs], e.g. for SMTP:
docker run --rm -ti containrrr/shoutrrr generate smtp
```

### Handle file uploads using third party tools

If you want to use a non-supported storage backend, or want to use a third party (e.g. rsync, rclone) tool for file uploads, you can build a Docker image containing the required binaries off this one, and call through to these in lifecycle hooks.

For example, if you wanted to use `rsync`, define your Docker image like this:

```Dockerfile
ARG version=canary
FROM offen/docker-volume-backup:$version

RUN apk add rsync
```

Using this image, you can now omit configuring any of the supported storage backends, and instead define your own mechanism in a `docker-volume-backup.copy-post` label:

```yml
version: '3'

services:
backup:
image: your-custom-image
restart: always
environment:
BACKUP_FILENAME: "daily-backup-%Y-%m-%dT%H-%M-%S.tar.gz"
BACKUP_CRON_EXPRESSION: "0 2 * * *"
labels:
- docker-volume-backup.copy-post=/bin/sh -c 'rsync $$COMMAND_RUNTIME_ARCHIVE_FILEPATH /destination'
volumes:
- app_data:/backup/app_data:ro
- /var/run/docker.sock:/var/run/docker.sock

# other services defined here ...
volumes:
app_data:
```


Commands will be invoked with the filepath of the tar archive passed as `COMMAND_RUNTIME_BACKUP_FILEPATH`.

## Recipes

This section lists configuration for some real-world use cases that you can mix and match according to your needs.
Expand Down
4 changes: 4 additions & 0 deletions cmd/backup/exec.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,14 @@ import (

func (s *script) exec(containerRef string, command string) ([]byte, []byte, error) {
args, _ := argv.Argv(command, nil, nil)
commandEnv := []string{
fmt.Sprintf("COMMAND_RUNTIME_ARCHIVE_FILEPATH=%s", s.file),
}
execID, err := s.cli.ContainerExecCreate(context.Background(), containerRef, types.ExecConfig{
Cmd: args[0],
AttachStdin: true,
AttachStderr: true,
Env: commandEnv,
})
if err != nil {
return nil, nil, fmt.Errorf("exec: error creating container exec: %w", err)
Expand Down
4 changes: 4 additions & 0 deletions test/extend/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
ARG version=canary
FROM offen/docker-volume-backup:$version

RUN apk add rsync
26 changes: 26 additions & 0 deletions test/extend/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
version: '3'

services:
backup:
image: offen/docker-volume-backup:${TEST_VERSION:-canary}
restart: always
labels:
- docker-volume-backup.copy-post=/bin/sh -c 'mkdir -p /tmp/unpack && tar -xvf $$COMMAND_RUNTIME_ARCHIVE_FILEPATH -C /tmp/unpack && rsync -r /tmp/unpack/backup/app_data /local'
environment:
BACKUP_FILENAME: test.tar.gz
BACKUP_CRON_EXPRESSION: 0 0 5 31 2 ?
EXEC_FORWARD_OUTPUT: "true"
volumes:
- ./local:/local
- app_data:/backup/app_data:ro
- /var/run/docker.sock:/var/run/docker.sock

offen:
image: offen/offen:latest
labels:
- docker-volume-backup.stop-during-backup=true
volumes:
- app_data:/var/opt/offen

volumes:
app_data:
28 changes: 28 additions & 0 deletions test/extend/run.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
#!/bin/sh

set -e

cd "$(dirname "$0")"
. ../util.sh
current_test=$(basename $(pwd))

mkdir -p local

export TEST_VERSION="${TEST_VERSION:-canary}-with-rsync"

docker build . -t offen/docker-volume-backup:$TEST_VERSION

docker-compose up -d
sleep 5

docker-compose exec backup backup

sleep 5

expect_running_containers "2"

if [ ! -f "./local/app_data/offen.db" ]; then
fail "Could not find expected file in untared archive."
fi

docker-compose down --volumes