WIP: Containerized secret server #96
Conversation
This feature allows you to e.g. access another docker containers from within a build for fetching build assets. See [this comment](cloud66-oss#80 (comment)) why this is implemented as a flag rather than a key in build.yml. Also see [the description about `--network` in the docker engine doc](https://docs.docker.com/engine/reference/commandline/build/) for more information, especially about why it is "networking mode" OR "network". Resolves cloud66-oss#80 This is verified manually by running habitus with the new example at exampls/network/build.yml: ``` $ ~/bin/habitus --build host=$host --build port=80 --network mynetwork 2018/03/15 20:39:27 ▶ Using '/Users/kuoka-yusuke/go/src/github.com/cloud66/habitus/examples/network/build.yml' as build file 2018/03/15 20:39:27 ▶ Collecting artifact information 2018/03/15 20:39:27 ▶ Building 1 steps 2018/03/15 20:39:27 ▶ Step 1 - builder, image-name = 'builder' 2018/03/15 20:39:27 ▶ Step 1 - Build for builder 2018/03/15 20:39:27 ▶ Step 1 - Building builder from context '/Users/kuoka-yusuke/go/src/github.com/cloud66/habitus/examples/network' 2018/03/15 20:39:27 ▶ Step 1 - Parsing and converting 'Dockerfile' 2018/03/15 20:39:27 ▶ Step 1 - Writing the new Dockerfile into '/Users/kuoka-yusuke/go/src/github.com/cloud66/habitus/examples/network/Dockerfile.generated' 2018/03/15 20:39:27 ▶ Step 1 - Building the builder image from /Users/kuoka-yusuke/go/src/github.com/cloud66/habitus/examples/network/Dockerfile.generated Step 1/7 : FROM ubuntu ---> f975c5035748 Step 2/7 : RUN apt-get update && apt-get install -y wget ---> Using cache ---> 07821c8ff8f6 Step 3/7 : ARG host ---> Using cache ---> 3fab8034e46a Step 4/7 : ARG port ---> Using cache ---> 004ee59f9a02 Step 5/7 : ENV ASSET /asset ---> Using cache ---> 20e3d5b78f2d Step 6/7 : RUN wget -q -O $ASSET http://$host:$port/ ---> Running in ad7137afc4cb ---> f94330eacbcd Removing intermediate container ad7137afc4cb Step 7/7 : RUN cat $ASSET ---> Running in c7cd838be46d <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html> ---> ecd4a27d7b4d Removing intermediate container c7cd838be46d Successfully built ecd4a27d7b4d Successfully tagged builder:latest ```
mumoshu
force-pushed
the
containerized-secret-server
branch
from
e31e947 to
616600f
Compare
|
Can it not be achieved by running a secondary process inside of the same container instead of a separate container? That way the secret server will only be available as the local loop (127.0.0.1) and not to the outside world in any form. My concern is from 2 aspects:
|
|
@khash Thank you so much for review comments! Your concerns sounds very valid to me. On the other hand, I'm sure if there's any better alternative than my suggested design.
I wish we could! We don't have a way to actually achieve that - docker doesn't allow us to inject an additional "sidecar" process along with its binary to the build container created implicitly via Ideally, it would be possible if docker provided an abstraction like a set of (side)containers sharing the loopback i/f like Kubernetes pods. However, in current docker it would be best emulated with a secret server container plus a build container sharing a docker network, probably? |
|
@mumoshu can we make this not merge into
|
|
@khash Sure! Please leave this PR not merged until I finish it, if that's what you meant. |
|
Sure. No problem. Thanks for letting me know. |
|
Hi @mumoshu Have you made any progress on this one? |
Depends on #94
Resolves #95
If anyone is interested in reviewing this, please see commits only added on top of the dependency.
Goals
In nutshell, I'll address the following items in this PR:
habitus serverto be run inside a container to serve the secret API in a specific docker networkapi.Serverimpl which delegates actual execution of the secret API server todocker run-containerize-api-serverwhich enables the containerized secret server(and on the other hand disables the traditional, embedded secret server)Synopsis
Either of the followings:
habitus --containerize-api-serverdocker run --network mynetwork cloud66/habitus:v1.x.x habitus server -foo -barto run the secret server in the docker network namedmynetworkand thenhabitus --network mynetworkto run a habitus build inside the network.Random notes on design and implementation
-server-timeout=600sto instruct the secret server container to suicide after 10min would be helpful to not leave the secret server running, which introduces an extra chance of secret breach.