Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add Dockerfile and Docker build Workflow #14

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

duhow
Copy link
Contributor

@duhow duhow commented Feb 26, 2025

Build a Docker image using the .deb installer, and add a small entrypoint script to start the server, and download the DB if not previously available.

docker build -t hibp -f docker/Dockerfile .

docker run -p 8082:8082 -e DATABASE=binfuse16 -v $PWD/data:/data hibp

@oschonrock
Copy link
Owner

oschonrock commented Feb 26, 2025

I don't have a lot of docker experience, so this is appreciated.

Would it make sense to add a docker system test?

If you have a look at test/system_tests.sh we run a local "mock" server server with a minimal dataset, that would allow the docker download to be tested, if you add a --testing to the hibp-download to use the local server.

then run the docker image and confirm queries work.

I guess that would require all devs who run the tests to have docker installed? Not great? What is good practice here?

also using the binary package means we are not testing the latest code. That means this "docker test" is a containerisation test only, not a code test. Therefore should be a separate test, which is run separately from test/system_test.sh ?

@duhow
Copy link
Contributor Author

duhow commented Feb 26, 2025

Hi! In that case, you are testing the binary build itself; packing it into Docker will behave exactly as built, so running tests will just generate the same duplicated result, no differences.

There are multiple approaches to follow as you want:

  • you can build binary after merging code, run tests, and if passes, then build a Docker image and publish as latest version, in a automatic way with GitHub Actions
  • you can build the binary itself within Docker, with different gcc versions, Python, or any deps you want, and run the tests there
  • trigger Docker build on release / git tag...

Right now, the Docker implementation just picks the .deb package (which should be already stable and tested), but I can implement the build as well.

@oschonrock
Copy link
Owner

Yes understand all that.

The code is well tested on a range of platforms. The only thing that needs additional test is the containerisation.

What is your intended use case for the docker image?
"An easy way of deploying a working hibp server" right?

is there a sensible way to test that the docker image does what it says on the tin?

I guess writing the docs for what the docker image does, should specify what the test should be?

@duhow
Copy link
Contributor Author

duhow commented Feb 26, 2025

With f271258 now it will build locally, requires having git submodule initialized as well.
Can pass docker build --build-arg TEST=OFF if needed.

@duhow
Copy link
Contributor Author

duhow commented Feb 26, 2025

The idea of having it containerized, is basically to pull the app and start the service quick and easy, no need to deal with package installation. End goal is to deploy in servers or Raspberry, as simple as docker run ghcr.io/oschonrock/hibp:0.6.1 .

@duhow
Copy link
Contributor Author

duhow commented Feb 26, 2025

Let me prepare a couple of scripts then...

@oschonrock
Copy link
Owner

right ok.. I agree with that. I als agree that raspberry pi and also firewall/openwall/router type devices are interesting deployment platforms.

raspberry pi is arm, so the x64 .deb won't work, so recompiling from source is required. But would it not make more sense to add raspberry pi as a supported hardware platform first, on which the github actions build in CI, and then add a docker wrapper? dealing with build errors through another layer of abstraction prob doesn't help.

So it comes back to what is the purpose of the docker image. Which platforms etc.. ? What's your view?

@duhow duhow marked this pull request as draft February 26, 2025 20:12
@duhow duhow marked this pull request as ready for review February 27, 2025 13:56
@duhow
Copy link
Contributor Author

duhow commented Feb 27, 2025

The workflow now will build the Docker image on merge (push to main) or git tag.
As example I have https://github.com/duhow/hibp/pkgs/container/hibp - may be deleted in future 🔥 , this is just an example.

There is currently a build issue with arm64, so I've disabled temporally the build for it in Docker, but can be enabled after the bug is addressed in a separate PR.

@oschonrock
Copy link
Owner

arm build should now hopefully work

just update your fork

@duhow
Copy link
Contributor Author

duhow commented Feb 27, 2025

image
Working!

@duhow duhow changed the title feat: add Dockerfile feat: add Dockerfile and Docker build Workflow Feb 27, 2025
Copy link
Owner

@oschonrock oschonrock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall this is looking great. thanks.

it will make deploying hibp-server much easier on a range of platforms, which is what we want.

if you could address the line specific comments/questions, and maybe add some readme docs in the "getting started and/or build" section. that would be great/

# Capture TERM signal and execute function
trap _term SIGTERM

hibp-server --bind-address 0.0.0.0 ${ARGS} ${EXTRA_ARGS} &
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should be binding to all addresses...

this is http without SSL. I think by default it should bind to localhost (which is the default for hibp-server).


# Partial database download
if [ ! -f "$PWD/data/hibp_all.sha1.bin" ]; then
$DOCKER_RUN $IMAGE_NAME hibp-download hibp_all.sha1.bin --limit 256 --no-progress
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I like that it's partial (--limit 256) for a test.

as I tried to explain earlier, we have a "tiny local cache" already committed to the repo. It is used in test/system_test.sh it is served with a custom restinio server

that cache takes the place of the online API from api.pwnedpasswords.com

it can be started as seen in system_test.sh and the download then needs to have --testing on it.

do you agree this is preferable as the docker test is then not reliant on some online resource and can be rune entirely locally?

@@ -0,0 +1,67 @@
#!/bin/bash
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume the idea that this will not be run during Ctest (at end of cmake build), because it it probably a bit heavy for that?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants