Python3.11.2 "Failed to initialise config" with 1.33.0-minimal

[kyzer-davis]

Problem

While attempting to build a Multi-language app and convert my working 1.31.1-python3.11 containers into Python3.11/FastAPI + Node/React multi-language containers; I found that Python3.11.2 always fails to load properly in 1.33.0-minimal. (See log below)

I have a sample repo here that one can clone and observe the issue:
https://github.com/kyzer-davis/nginx-unit-python-multi-app

Simply issue docker compose up --build --remove-orphans --force-recreate -d

Notes:

• The Log line that indicates the failure to initialize was added via Failed to initialise config: 491d0f7 but really does not provide anything useful.
  • Seeing pep405, I have tried with and without VENV's but didn't find much difference.
• I realized my working container (see "Working Single-App Container" section) is running python 3.11.8 so I pulled that and built from source.
  • 3.11.8 seems to get past the error however, I run into problems with Unit loading installed Python modules (likely because Unit is 3.11.2 and Python/Pip are 3.11.8.)

My analysis, it looks like the problem may be with 3.11.2 since 3.11.8 load nicely.

• I can compile and build 3.11.8 from Python source but the unit-python3.11 is 3.11.2 (likely since Debian python3.11 is currently only at 3.11.2) so I doubt this will work until the Debian package updates to a newer version of 3.11.
• I am willing to try Python3.12 but I don't see a unit-python3.12 package for Bookwork which is what 1.33.0-minmal runs right now..

---

Versions

root@27f4af62d939:/# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

root@27f4af62d939:/# which python3
/usr/bin/python3
root@27f4af62d939:/# which python3.11
/usr/bin/python3.11
root@27f4af62d939:/# python3 --version
Python 3.11.2

Configs

root@27f4af62d939:/# unitd --version
unit version: 1.33.0
configured as ./configure --prefix=/usr --statedir=/var/lib/unit --control=unix:/var/run/control.unit.sock --runstatedir=/var/run --pid=/var/run/unit.pid --logdir=/var/log --log=/var/log/unit.log --tmpdir=/var/tmp --user=unit --group=unit --tests --openssl --njs --modulesdir=/usr/lib/unit/modules --libdir=/usr/lib/x86_64-linux-gnu --cc-opt='-g -O2 -ffile-prefix-map=/data/builder/debuild/unit-1.33.0/pkg/deb/debuild/unit-1.33.0=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --ld-opt='-specs=/usr/share/dpkg/no-pie-link.specs -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie

LOGS:

root@ubuntu-web-server-2:/nginx-unit-python-multi-app# docker compose logs
nginx-unit-python  | /usr/local/bin/docker-entrypoint.sh: Launching Unit daemon to perform initial configuration...
nginx-unit-python  | 2024/11/27 18:41:17 [info] 9#9 unit 1.33.0 started
nginx-unit-python  | 2024/11/27 18:41:17 [info] 12#12 discovery started
nginx-unit-python  | 2024/11/27 18:41:17 [notice] 12#12 module: python 3.11.2 "/usr/lib/unit/modules/python3.11.unit.so"
nginx-unit-python  | 2024/11/27 18:41:17 [info] 10#10 controller started
nginx-unit-python  | 2024/11/27 18:41:17 [notice] 10#10 process 12 exited with code 0
nginx-unit-python  | 2024/11/27 18:41:17 [info] 15#15 router started
nginx-unit-python  | 2024/11/27 18:41:17 [info] 15#15 OpenSSL 3.0.15 3 Sep 2024, 300000f0
nginx-unit-python  | {
nginx-unit-python  |    "certificates": {},
nginx-unit-python  |    "js_modules": {},
nginx-unit-python  |    "config": {
nginx-unit-python  |            "listeners": {},
nginx-unit-python  |            "routes": [],
nginx-unit-python  |            "applications": {}
nginx-unit-python  |    },
nginx-unit-python  |
nginx-unit-python  |    "status": {
nginx-unit-python  |            "modules": {
nginx-unit-python  |                    "python": {
nginx-unit-python  |                            "version": "3.11.2",
nginx-unit-python  |                            "lib": "/usr/lib/unit/modules/python3.11.unit.so"
nginx-unit-python  |                    }
nginx-unit-python  |            },
nginx-unit-python  |
nginx-unit-python  |            "connections": {
nginx-unit-python  |                    "accepted": 0,
nginx-unit-python  |                    "active": 0,
nginx-unit-python  |                    "idle": 0,
nginx-unit-python  |                    "closed": 0
nginx-unit-python  |            },
nginx-unit-python  |
nginx-unit-python  |            "requests": {
nginx-unit-python  |                    "total": 0
nginx-unit-python  |            },
nginx-unit-python  |
nginx-unit-python  |            "applications": {}
nginx-unit-python  |    }
nginx-unit-python  | }
nginx-unit-python  | /usr/local/bin/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, applying initial configuration...
nginx-unit-python  | /usr/local/bin/docker-entrypoint.sh: Looking for certificate bundles in /docker-entrypoint.d/...
nginx-unit-python  | /usr/local/bin/docker-entrypoint.sh: Uploading certificates bundle: /docker-entrypoint.d/fullchain_rsa.pem
nginx-unit-python  | /usr/local/bin/docker-entrypoint.sh: OK: HTTP response status code is '200'
 }success": "Certificate chain uploaded."
nginx-unit-python  | /usr/local/bin/docker-entrypoint.sh: Looking for JavaScript modules in /docker-entrypoint.d/...
nginx-unit-python  | /usr/local/bin/docker-entrypoint.sh: Looking for configuration snippets in /docker-entrypoint.d/...
nginx-unit-python  | /usr/local/bin/docker-entrypoint.sh: Applying configuration /docker-entrypoint.d/config.json
nginx-unit-python  | 2024/11/27 18:41:17 [info] 30#30 "fastapi" prototype started
nginx-unit-python  | 2024/11/27 18:41:17 [info] 31#31 "fastapi" application started
nginx-unit-python  | Python path configuration:
nginx-unit-python  |   PYTHONHOME = '/usr/local'
nginx-unit-python  |   PYTHONPATH = (not set)
nginx-unit-python  |   program name = 'python3'
nginx-unit-python  |   isolated = 1
nginx-unit-python  |   environment = 0
nginx-unit-python  |   user site = 0
nginx-unit-python  |   safe_path = 1
nginx-unit-python  |   import site = 1
nginx-unit-python  |   is in build tree = 0
nginx-unit-python  |   stdlib dir = '/usr/local/lib/python3.11'
nginx-unit-python  |   sys._base_executable = '/usr/bin/python3'
nginx-unit-python  |   sys.base_prefix = '/usr/local'
nginx-unit-python  |   sys.base_exec_prefix = '/usr/local'
nginx-unit-python  |   sys.platlibdir = 'lib'
nginx-unit-python  |   sys.executable = '/usr/bin/python3'
nginx-unit-python  |   sys.prefix = '/usr/local'
nginx-unit-python  |   sys.exec_prefix = '/usr/local'
nginx-unit-python  |   sys.path = [
nginx-unit-python  |     '/usr/local/lib/python311.zip',
nginx-unit-python  |     '/usr/local/lib/python3.11',
nginx-unit-python  |     '/usr/local/lib/python3.11/lib-dynload',
nginx-unit-python  |   ]
nginx-unit-python  | 2024/11/27 18:41:17 [alert] 31#31 Failed to initialise config
nginx-unit-python  | 2024/11/27 18:41:17 [notice] 30#30 app process 31 exited with code 1
nginx-unit-python  | 2024/11/27 18:41:17 [warn] 15#15 failed to start application "fastapi"
nginx-unit-python  | 2024/11/27 18:41:17 [alert] 15#15 failed to apply new conf
nginx-unit-python  | 2024/11/27 18:41:17 [notice] 10#10 process 30 exited with code 0
nginx-unit-python  | /usr/local/bin/docker-entrypoint.sh: Error: HTTP response status code is '500'
 }error": "Failed to apply new configuration."
nginx-unit-python  | /usr/local/bin/docker-entrypoint.sh: /var/lib/unit/ is not empty, skipping initial configuration...
nginx-unit-python  | 2024/11/27 18:41:17 [info] 1#1 unit 1.33.0 started
nginx-unit-python  | 2024/11/27 18:41:17 [info] 9#9 discovery started
nginx-unit-python  | 2024/11/27 18:41:17 [notice] 9#9 module: python 3.11.2 "/usr/lib/unit/modules/python3.11.unit.so"
nginx-unit-python  | 2024/11/27 18:41:17 [info] 1#1 controller started
nginx-unit-python  | 2024/11/27 18:41:17 [notice] 1#1 process 9 exited with code 0
nginx-unit-python  | 2024/11/27 18:41:17 [info] 11#11 router started
nginx-unit-python  | 2024/11/27 18:41:17 [info] 11#11 OpenSSL 3.0.15 3 Sep 2024, 300000f0

Debian and Unit Python3

root@27f4af62d939:/# apt search "^unit-python"
Sorting... Done
Full Text Search... Done
unit-python3.11/now 1.33.0-1~bookworm amd64 [installed,local]
  Python 3.11 module for NGINX Unit

root@27f4af62d939:/# ls -la "/usr/lib/unit/modules"
total 148
drwxr-xr-x 1 root root   4096 Nov 27 18:39 .
drwxr-xr-x 1 root root   4096 Nov 12 02:21 ..
-rw-r--r-- 1 root root 138600 Sep 17 21:42 python3.11.unit.so

---

Working Single-App Container

FROM unit:1.31.1-python3.11

root@6edc0a01023e:/# which python3
/usr/local/bin/python3
root@6edc0a01023e:/# python3 --version
Python 3.11.8

root@6edc0a01023e:/# unitd --version
unit version: 1.31.1
configured as ./configure --prefix=/usr --statedir=/var/lib/unit --control=unix:/var/run/control.unit.sock --runstatedir=/var/run --pid=/var/run/unit.pid --logdir=/var/log --log=/var/log/unit.log --tmpdir=/var/tmp --user=unit --group=unit --openssl --libdir=/usr/lib/x86_64-linux-gnu --njs --cc-opt='-g -O2 -ffile-prefix-map=/usr/src/unit/unit=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --ld-opt='-specs=/usr/share/dpkg/no-pie-link.specs -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie' --modulesdir=/usr/lib/unit/modules

root@6edc0a01023e:/# ls -la "/usr/lib/unit/modules"
total 820
drwxr-xr-x 2 root root   4096 Feb 13  2024 .
drwxr-xr-x 4 root root   4096 Feb 13  2024 ..
-rwxr-xr-x 1 root root 827728 Feb 13  2024 python3.unit.so

root@ubuntu-web-server-2:/# docker compose logs | grep .unit.so
python-app  | 2024/11/13 18:28:18 [notice] 12#12 module: python 3.11.8 "/usr/lib/unit/modules/python3.unit.so"

[kyzer-davis]

Further, I do not believe the current documentation is up-to-date for 1.33.0: https://unit.nginx.org/howto/docker/#multilanguage-images

For example, I don't see unit-python3.9 available for Bookworm. (I don't want this, just illustrating how the document snippet is out of date)

root@27f4af62d939:/# apt search "^unit-python"
Sorting... Done
Full Text Search... Done
unit-python3.11/now 1.33.0-1~bookworm amd64 [installed,local]
  Python 3.11 module for NGINX Unit

One may also be able to reference the setup-unit tool for the repo config as seen on https://unit.nginx.org/installation/#official-packages > "Repo installation script" rather than what is in the docs right now.
Current: https://github.com/kyzer-davis/nginx-unit-python-multi-app/blob/main/Dockerfile#L3-L7
New: https://github.com/kyzer-davis/nginx-unit-python-multi-app/blob/main/Dockerfile#L9-L12