Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for local secrets #227

Merged
merged 11 commits into from
Jul 2, 2024
+38 −18
Merged

Add support for local secrets #227

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
00db98b
Impl
KevinJBoyer Jun 14, 2024
be5f5b5
Add .gitignore
KevinJBoyer Jun 14, 2024
870ed31
Move files and update docs
KevinJBoyer Jun 24, 2024
709f536
Merge branch 'main' into kb/secrets-in-docker
KevinJBoyer Jun 24, 2024
90c4ab4
Remove version from override example
KevinJBoyer Jun 24, 2024
31a0c22
Update paths in compose files
KevinJBoyer Jun 24, 2024
5703d6a
Fix mount dest
KevinJBoyer Jun 24, 2024
937a6a7
Patch documentation
KevinJBoyer Jun 28, 2024
a145f07
Merge branch 'main' into kb/secrets-in-docker
KevinJBoyer Jun 28, 2024
f077472
Fake commit for CI
KevinJBoyer Jul 2, 2024
2dd4a10
Undo fake commit for CI
KevinJBoyer Jul 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions app/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,3 +29,8 @@ coverage.*

# Poetry installer local error logs
poetry-installer-error-*.log

# This file is used in local development to pass an /app/.env
# file to the container, for secrets. It should not be committed
# to the repo because tests and CI/CD will not have an .env file.
docker-compose.override.yml
6 changes: 3 additions & 3 deletions docker-compose.debug.yml → app/docker-compose.debug.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@
services:
main-app:
build:
context: ./app
context: ./
target: dev
args:
- RUN_UID=${RUN_UID:-4000}
- RUN_USER=${RUN_USER:-app}
container_name: main-app
env_file: ./app/local.env
env_file: local.env
command: [
"poetry", "run", "python", "-m", "debugpy",
"--listen", "0.0.0.0:5678",
Expand All @@ -20,4 +20,4 @@ services:
ports:
- 5678:5678
volumes:
- ./app:/app
- ./:/app
6 changes: 6 additions & 0 deletions app/docker-compose.override.yml.example
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
services:

main-app:
env_file:
- local.env
- .env
6 changes: 3 additions & 3 deletions docker-compose.yml → app/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,18 +17,18 @@ services:

main-app:
build:
context: ./app
context: ./
target: dev
args:
- RUN_UID=${RUN_UID:-4000}
- RUN_USER=${RUN_USER:-app}
command: ["poetry", "run", "flask", "--app", "src.app", "run", "--host", "0.0.0.0", "--port", "8080", "--reload"]
container_name: main-app
env_file: ./app/local.env
env_file: local.env
ports:
- 8080:8080
volumes:
- ./app:/app
- ./:/app
depends_on:
- main-db

Expand Down
4 changes: 4 additions & 0 deletions app/local.env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,8 +63,12 @@ HIDE_SQL_PARAMETER_LOGS=TRUE
# add them to this file to avoid mistakenly
# committing them. Set these in your shell
# by doing `export AWS_ACCESS_KEY_ID=whatever`
# if you are running the app directly, or
# in your `app/.env` if you are running the
# app in a Docker container
Comment on lines +66 to +68
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Threading arbitrarily, - we might also need a change to the load_local_env_vars method which handles getting the local.env file into your env vars when you run outside of docker. Right now that automatically gets called for unit tests in the conftest file:

https://github.com/navapbc/template-application-flask/blob/main/app/tests/conftest.py#L21

I think if we just adjusted load_local_env_vars to do "if .env exists, load it before the local.env file" bit - then it would work.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively - we could get rid of load_local_env_vars entirely and more accurately document how to do that properly when running outside of docker. Maybe I'll take that on next week.

AWS_ACCESS_KEY_ID=DO_NOT_SET_HERE
AWS_SECRET_ACCESS_KEY=DO_NOT_SET_HERE

# These next two are commented out as we
# don't have configuration for individuals
# to use these at the moment and boto3
Expand Down
26 changes: 16 additions & 10 deletions docs/app/getting-started.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ A very simple [docker-compose.yml](/docker-compose.yml) has been included to sup

## Prerequisites

**Note:** Run everything from within the `/app` folder:

1. Install the version of Python specified in [.python-version](/app/.python-version)
[pyenv](https://github.com/pyenv/pyenv#installation) is one popular option for installing Python,
or [asdf](https://asdf-vm.com/).
Expand All @@ -21,17 +19,25 @@ A very simple [docker-compose.yml](/docker-compose.yml) has been included to sup

3. If you are using an M1 mac, you will need to install postgres as well: `brew install postgresql` (The psycopg2-binary is built from source on M1 macs which requires the postgres executable to be present)

4. You'll also need [Docker Desktop](https://www.docker.com/products/docker-desktop/)
4. You'll also need [Docker Desktop](https://www.docker.com/products/docker-desktop/) installed and running.

## Run the application

1. In your terminal, `cd` to the `app` directory of this repo.
2. Make sure you have [Docker Desktop](https://www.docker.com/products/docker-desktop/) installed & running.
3. Run `make setup-local` to install dependencies
Comment on lines -28 to -30
Copy link
Contributor Author

@KevinJBoyer KevinJBoyer Jun 24, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed these lines:

  • 1 is redundant with the note about running from within the app folder (which I also moved to be directly above these steps)
  • 2 is redundant with the prerequisites above. If someone does miss that note, the error message is very clear ("Is Docker running?")
  • I believe 3 is not necessary since these steps assume running everything in the Docker container. For folks who do want to run things like formatting, etc, directly on their machine, the linked application docs under Next steps at the end of this do a great job explaining how to do so (including doing this step)

4. Run `make init start` to build the image and start the container.
5. Navigate to `localhost:8080/docs` to access the Swagger UI.
6. Run `make run-logs` to see the logs of the running API container
7. Run `make stop` when you are done to delete the container.
**Note:** Run everything from within the `/app` folder:

1. Run `make init start` to build the image and start the container.
2. Navigate to `localhost:8080/docs` to access the Swagger UI.
3. Run `make run-logs` to see the logs of the running API container
4. Run `make stop` when you are done to delete the container.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line is a bit misleading. Deleting might make someone think that the contents of their container will be removed when they run stop, but that isn't true as that's cached and/or stored in the volume.

Maybe we should adjust this to something like .. when you are done to stop the container.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 will make this change!


## (Optional) Configure local secrets

If you need to pass secrets to the application via environment variables, copy the provided [/app/docker-compose.override.yml.example](/docker-compose.override.yml.example) to `/app/docker-compose.override.yml`. Then create an `/app/.env` file with your secrets. The override will pass this file to the Docker container with your application.

```bash
cp docker-compose.override.yml.example docker-compose.override.yml
touch app/.env
```

## Next steps

Expand Down
Loading