Example Docker/container setup

[scy]

We know that DearMEP isn't exactly super easy to set up, and that reading through the documentation can be somewhat daunting. And some people prefer reading code instead of walls of text.

Which is why I've put together an example setup of DearMEP from scratch that you can run to quickly get a working version. It doesn't come with phone functionality preconfigured (because you'd need a 46elks account), and it doesn't replace the default secrets in the config file (which you should definitely do on a production instance), but it demonstrates how to collect the data it needs to run.

FROM python:3.9-alpine

RUN apk add --no-cache \
	curl \
	ffmpeg \
	gcc \
	git \
	linux-headers \
	musl-dev \
	python3-dev \
	rsync \
	zstd

RUN pip install 'dearmep[convert]'

# You can of course use a different directory, but this is the default in the configs.
RUN mkdir /var/dearmep
WORKDIR /var/dearmep

# Serve a demo page on `/index.html`.
ENV DEARMEP_DEMO_PAGE=y
# Serve Markdown files; the contents will be downloaded in the setup script.
ENV DEARMEP_MARKDOWN_FILES_DIR=/var/dearmep/markdown

CMD dearmep serve --host 0.0.0.0
EXPOSE 8000

And here's a script that you can run in the container to populate the database.

#!/bin/sh
set -e

# DEARMEP_MARKDOWN_FILES_DIR is set in the container and will need to exist for
# DearMEP to start.
mkdir -p markdown

# Generate an example config. On a production instance, you should replace the
# secrets `authentication.secrets.pepper` and `authentication.secrets.jwt.key`.
dearmep dump example-config > config.yaml

if ! [ -e repo ]; then
	# Clone the main repo for the example markdown files.
	git clone https://github.com/AKVorrat/dearmep.git repo
fi
if ! [ -e media ]; then
	# Clone the media repo for group logos and IVR audio.
	git clone https://github.com/AKVorrat/dearmep-media.git media
fi

# Copy example markdown in place.
rsync -rtl repo/server/example-markdown/ markdown

if ! [ -e ep_meps.json ]; then
	# Fetch the list of MEPs from Parltrack.
	curl -L https://parltrack.org/dumps/ep_meps.json.zst | zstdcat > ep_meps.json
fi

if ! [ -e dearmep-destinations.json ]; then
	# Convert the Parltrack dump to our own NDJSON format.
	dearmep convert parltrack.meps --no-lz ep_meps.json > dearmep-destinations.json
fi

# Download portraits from the European Parliament.
mkdir -p portraits
in2csv -f ndjson dearmep-destinations.json \
	| csvgrep -c _dearmep_type -m destination \
	| csvcut -c id \
	| tail -n +2 \
	| xargs dearmep convert europarl.portraits --filename-template portraits/'{id}.jpg' --not-found ignore --existing skip
dearmep convert europarl.portraits --filename-template portraits/placeholder.jpg --not-found save --existing skip 0

# Download audio versions of the MEP names from the European Parliament.
mkdir -p name-audio
in2csv -f ndjson dearmep-destinations.json \
	| csvgrep -c _dearmep_type -m destination \
	| csvcut -c id \
	| tail -n +2 \
	| xargs dearmep convert europarl.name-audio --filename-template name-audio/'{id}.mp3' --not-found ignore --existing skip

# Convert the name MP3s and IVR WAVs to Vorbis.
dearmep convert audio --existing skip name-audio/*.mp3 media/ivr/*.wav

# Drop the existing database and initialize it from scratch. (Don't do this on a
# production instance of course.)
rm -f dearmep.sqlite
dearmep alembic upgrade head

# Insert all the IVR audio into the database.
dearmep db store-blob --type ivr_audio --mime audio/ogg media/ivr/*.ogg

# Import the MEPs with associated data.
dearmep import destinations --portrait-template 'portraits/{filename}' --fallback-portrait portraits/placeholder.jpg --logo-template 'media/logos/{filename}' --name-audio-template 'name-audio/{filename}' dearmep-destinations.json

# Generate random swayability values and import these.
in2csv -f ndjson dearmep-destinations.json | csvgrep -c _dearmep_type -m destination | csvcut -c id | csvsql --no-inference --query 'select id, abs(random()) % 101 / 100.0 as endorsement from stdin' | dearmep import swayability

Let me know if anything isn't working or you have questions!