diff --git a/nixos/doc/manual/development/development.xml b/nixos/doc/manual/development/development.xml index 0b2ad60a878b07..45a39db8685890 100644 --- a/nixos/doc/manual/development/development.xml +++ b/nixos/doc/manual/development/development.xml @@ -11,6 +11,7 @@ + diff --git a/nixos/doc/manual/development/systemd-additional-resources.section.md b/nixos/doc/manual/development/systemd-additional-resources.section.md new file mode 100644 index 00000000000000..8f1ddab04ed43e --- /dev/null +++ b/nixos/doc/manual/development/systemd-additional-resources.section.md @@ -0,0 +1,6 @@ +## Additional resources {#sec-systemd-additional-resources} + +[Systemd official website](https://systemd.io/) +[Wikipedia](https://en.wikipedia.org/wiki/Systemd) +[Archlinux Wiki](https://wiki.archlinux.org/title/Systemd) +[Systemd announcement from Lennart Poettering (one of the 2 original developpers of systemd)](http://0pointer.de/blog/projects/systemd.html) diff --git a/nixos/doc/manual/development/systemd-environment-and-secrets.section.md b/nixos/doc/manual/development/systemd-environment-and-secrets.section.md new file mode 100644 index 00000000000000..42f68679e61102 --- /dev/null +++ b/nixos/doc/manual/development/systemd-environment-and-secrets.section.md @@ -0,0 +1,32 @@ +## Environment and secret maangement {#sec-systemd-environment} + +The `environment` attribute is used to pass environment variables to the systemd service. +``` +environment = { + MY_ENV_VAR = "Something blue"; +} +``` + +A file can be passed as well with `environmentFile`. If both are passed, `environmentFile` takes precedence over `environment` + +Environment variables should not contain secret data. +The main reason being that environment variables are accessible to unpriviledged users. +See [systemd doc](https://www.freedesktop.org/software/systemd/man/systemd.exec.html#Environment=) for a more detailed explanation. + +### Secret management {#sec-systemd-secrets} + +To pass secret data to a systemd service, systemd has the `LoadCredential` mechanism. +By passing a file path to systemd, it will make sure that the file is available to the process under a specific path. +The load credential attribute takes an id a colon and a file path +let's have a look at an [example](https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/web-apps/lemmy.nix#L182) +``` +LoadCredential = "jwt_secret:${cfg.jwtSecretPath}"; +``` +Inside your application, the jwt secret will be available under `$CREDENTIALS_DIRECTORY/jwt_secret` environment variable. + +Note that this requires your application to read secrets from a file. + +If your service can only read secrets through environment variables, the best alternative is to ask the end user to pass an `environmentFile` containing the secrets. + +If your service reads secrets from a configuration file, you can use the `script` attribute of your systemd service to populate that file with the secrets before starting the service. +[example in nixpkgs](https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/web-apps/lemmy.nix#L172) diff --git a/nixos/doc/manual/development/systemd-handling-dependencies.section.md b/nixos/doc/manual/development/systemd-handling-dependencies.section.md new file mode 100644 index 00000000000000..8563b34e2d08ca --- /dev/null +++ b/nixos/doc/manual/development/systemd-handling-dependencies.section.md @@ -0,0 +1,26 @@ +## Handling dependencies {#sec-systemd-handling-dependencies} + +When a systemd service A depends on another service B. +There are 3 choices to make +wants vs requires vs bindsTo: +- use `wants = [ "B" ];`, if service A can start even if B fails. Systemd will try to start B but will start A even in case of a failure of B. +- use `requires = [ "B" ];`, if service A will not start without B. Systemd will try to start B and will not start A in case B fails. +- use `bindsTo = [ "B" ];`, if service A should be shutdown in case of a failure of B. + +After: +- use `after = [ "B" ];`, if service A needs B to active before it is started. Note that with this setting, if you shutdown B, A will be shut down first. + +PartOf: +- use `partOf = [ "B" ];`, if service A needs to re-started when service B is. + +Note that `wants,requires,bindsTo` are orthogonal to `after`. The most common occurence in nixpkgs is `requires` and `after`. + +### Examples {#sec-systemd-handling-dependencies-examples} + +- Web-app dependency on a database. If the web-app cannot start without the database, use `requires` and `after`. If you would rather have it operate even without a database use `wants` and `after`. [example in nixpkgs](https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/web-apps/lemmy.nix#L169) +- Database migration and setup script for a web-app. It is common to make a service for database setup and migration. This service should be `partOf` the webapp service as everytime the webapp is restarted, this service should be restarted as well. +[example in nixpkgs](https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/web-apps/lemmy.nix#L214) +- A web-app has two separate services, a UI and a server. The ui `requires` `after` the server. [example in nixpkgs](https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/web-apps/lemmy.nix#L205) + + +Note there are many other settings, for a complete reference and more detailed explanations, see the [systemd manual section](https://man.archlinux.org/man/systemd.unit.5#%5BUNIT%5D_SECTION_OPTIONS) diff --git a/nixos/doc/manual/development/systemd-state-management.section.md b/nixos/doc/manual/development/systemd-state-management.section.md new file mode 100644 index 00000000000000..b254e53a73e732 --- /dev/null +++ b/nixos/doc/manual/development/systemd-state-management.section.md @@ -0,0 +1,24 @@ +## State management {#sec-systemd-state-management} + +State refers to files and directories for config or running a service. +A special section is dedicated at the end regarding databases. + +TODO: section on RuntimeDir, StateDir, WorkingDir + +### Database connection {#sec-systemd-state-management-database-connection} + +Most databases have 2 modes of connection, over TCP or over Unix socket. Connecting over a unix socket can provide 2 advantages +- 30% performance improvement as packets don't need to be encoder over TCP +- The default authentication mode postgres on unix sockets is just with the user name. This will remove the need to manage the password secret + +Therefore the default for database connection for services is recommended to be with a unix socket. For postgres for example, the `host` needs to start with a `/`. +[nixpkgs-example](https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/web-apps/lemmy.nix#L98) + +### Database initialisation {#sec-systemd-state-management-database-initialisation} + +To initialise a database that a service depends on, it's customary to create another systemd service that will be `partOf` your main service. +The initialisation service should be run by the database superuser so as not to require sudo. +Typically it will involve creating the database and the user. +There will be a step to see if the database is already created. +That check is best implemented by going throug the created tables and exiting if the table already exists. +[nixpkgs-example](https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/web-apps/lemmy.nix#L220) diff --git a/nixos/doc/manual/development/systemd.chapter.md b/nixos/doc/manual/development/systemd.chapter.md new file mode 100644 index 00000000000000..e3f3befa88504f --- /dev/null +++ b/nixos/doc/manual/development/systemd.chapter.md @@ -0,0 +1,51 @@ +# Systemd {#sec-systemd} + +Systemd is a building block of most nixos modules. + +From the [project's webpage](https://systemd.io/) +``` +systemd is a suite of basic building blocks for a Linux system. It provides a system and service manager that runs as PID 1 and starts the rest of the system. + +systemd provides aggressive parallelization capabilities, uses socket and D-Bus activation for starting services, offers on-demand starting of daemons, keeps track of processes using Linux control groups, maintains mount and automount points, and implements an elaborate transactional dependency-based service control logic. systemd supports SysV and LSB init scripts and works as a replacement for sysvinit. + +Other parts include a logging daemon, utilities to control basic system configuration like the hostname, date, locale, maintain a list of logged-in users and running containers and virtual machines, system accounts, runtime directories and settings, and daemons to manage simple network configuration, network time synchronization, log forwarding, and name resolution. +``` + +This chapter aims at describing the most commonly used features of systemd in nixos. + +Index + +- usage in modules + - Starting a service + [ref](https://man.archlinux.org/man/systemd.service.5.en) + - Handling dependencies + Talk about `requires` `wants` `after` `partOf` and the likes + [ref](https://man.archlinux.org/man/systemd.unit.5#%5BUNIT%5D_SECTION_OPTIONS) + - Environment + Talk about `environment` how to use it and not passing secrets + [ref](https://man.archlinux.org/man/systemd.exec.5.en#ENVIRONMENT) + - Managing state + Talk about StateDir RunDir WorkingDir and when to use each + - Initialisation of state and dependents (e.g. database) + Talk about adding a systemd service to handle database initialisation + - Database management + Talk about connection via a unix socket for more performance and reduced secret management + - Managing secrets + Talk about LoadCredential and how to use it + [ref](https://man.archlinux.org/man/systemd.exec.5.en#CREDENTIALS) + - Security and principle of least privilege + Talk about `DynamicUser` and user management + [ref](https://man.archlinux.org/man/systemd.exec.5.en#USER/GROUP_IDENTITY) + - Template units + Show an example and explain why you would use them + - Advanced topics + - Capabilities + [ref](https://man.archlinux.org/man/systemd.exec.5.en#CAPABILITIES) + - Socket activation for zero downtime deploys + +```{=docbook} + + + + +``` diff --git a/nixos/doc/manual/from_md/development/systemd-additional-resources.section.xml b/nixos/doc/manual/from_md/development/systemd-additional-resources.section.xml new file mode 100644 index 00000000000000..5c870c1115bab0 --- /dev/null +++ b/nixos/doc/manual/from_md/development/systemd-additional-resources.section.xml @@ -0,0 +1,13 @@ +
+ Additional resources + + Systemd official + website + Wikipedia + Archlinux + Wiki + Systemd + announcement from Lennart Poettering (one of the 2 original + developpers of systemd) + +
diff --git a/nixos/doc/manual/from_md/development/systemd-environment-and-secrets.section.xml b/nixos/doc/manual/from_md/development/systemd-environment-and-secrets.section.xml new file mode 100644 index 00000000000000..69f7be3c3c50e5 --- /dev/null +++ b/nixos/doc/manual/from_md/development/systemd-environment-and-secrets.section.xml @@ -0,0 +1,61 @@ +
+ Environment and secret maangement + + The environment attribute is used to pass + environment variables to the systemd service. + + +environment = { + MY_ENV_VAR = "Something blue"; +} + + + A file can be passed as well with + environmentFile. If both are passed, + environmentFile takes precedence over + environment + + + Environment variables should not contain secret data. The main + reason being that environment variables are accessible to + unpriviledged users. See + systemd + doc for a more detailed explanation. + +
+ Secret management + + To pass secret data to a systemd service, systemd has the + LoadCredential mechanism. By passing a file + path to systemd, it will make sure that the file is available to + the process under a specific path. The load credential attribute + takes an id a colon and a file path let’s have a look at an + example + + +LoadCredential = "jwt_secret:${cfg.jwtSecretPath}"; + + + Inside your application, the jwt secret will be available under + $CREDENTIALS_DIRECTORY/jwt_secret environment + variable. + + + Note that this requires your application to read secrets from a + file. + + + If your service can only read secrets through environment + variables, the best alternative is to ask the end user to pass an + environmentFile containing the secrets. + + + If your service reads secrets from a configuration file, you can + use the script attribute of your systemd + service to populate that file with the secrets before starting the + service. + example + in nixpkgs + +
+
diff --git a/nixos/doc/manual/from_md/development/systemd-handling-dependencies.section.xml b/nixos/doc/manual/from_md/development/systemd-handling-dependencies.section.xml new file mode 100644 index 00000000000000..8df40a4218a82a --- /dev/null +++ b/nixos/doc/manual/from_md/development/systemd-handling-dependencies.section.xml @@ -0,0 +1,99 @@ +
+ Handling dependencies + + When a systemd service A depends on another service B. There are 3 + choices to make wants vs requires vs bindsTo: + + + + + use wants = [ "B" ];, if service A + can start even if B fails. Systemd will try to start B but will + start A even in case of a failure of B. + + + + + use requires = [ "B" ];, if service + A will not start without B. Systemd will try to start B and will + not start A in case B fails. + + + + + use bindsTo = [ "B" ];, if service + A should be shutdown in case of a failure of B. + + + + + After: + + + + + use after = [ "B" ];, if service A + needs B to active before it is started. Note that with this + setting, if you shutdown B, A will be shut down first. + + + + + PartOf: + + + + + use partOf = [ "B" ];, if service A + needs to re-started when service B is. + + + + + Note that wants,requires,bindsTo are orthogonal + to after. The most common occurence in nixpkgs is + requires and after. + +
+ Examples + + + + Web-app dependency on a database. If the web-app cannot start + without the database, use requires and + after. If you would rather have it operate + even without a database use wants and + after. + example + in nixpkgs + + + + + Database migration and setup script for a web-app. It is + common to make a service for database setup and migration. + This service should be partOf the webapp + service as everytime the webapp is restarted, this service + should be restarted as well. + example + in nixpkgs + + + + + A web-app has two separate services, a UI and a server. The ui + requires after the + server. + example + in nixpkgs + + + + + Note there are many other settings, for a complete reference and + more detailed explanations, see the + systemd + manual section + +
+
diff --git a/nixos/doc/manual/from_md/development/systemd-state-management.section.xml b/nixos/doc/manual/from_md/development/systemd-state-management.section.xml new file mode 100644 index 00000000000000..e3ffd69fa31db1 --- /dev/null +++ b/nixos/doc/manual/from_md/development/systemd-state-management.section.xml @@ -0,0 +1,54 @@ +
+ State management + + State refers to files and directories for config or running a + service. A special section is dedicated at the end regarding + databases. + + + TODO: section on RuntimeDir, StateDir, WorkingDir + +
+ Database connection + + Most databases have 2 modes of connection, over TCP or over Unix + socket. Connecting over a unix socket can provide 2 advantages + + + + + 30% performance improvement as packets don’t need to be + encoder over TCP + + + + + The default authentication mode postgres on unix sockets is + just with the user name. This will remove the need to manage + the password secret + + + + + Therefore the default for database connection for services is + recommended to be with a unix socket. For postgres for example, + the host needs to start with a + /. + nixpkgs-example + +
+
+ Database initialisation + + To initialise a database that a service depends on, it’s customary + to create another systemd service that will be + partOf your main service. The initialisation + service should be run by the database superuser so as not to + require sudo. Typically it will involve creating the database and + the user. There will be a step to see if the database is already + created. That check is best implemented by going throug the + created tables and exiting if the table already exists. + nixpkgs-example + +
+
diff --git a/nixos/doc/manual/from_md/development/systemd.chapter.xml b/nixos/doc/manual/from_md/development/systemd.chapter.xml new file mode 100644 index 00000000000000..68b8e9cd485fac --- /dev/null +++ b/nixos/doc/manual/from_md/development/systemd.chapter.xml @@ -0,0 +1,114 @@ + + Systemd + + Systemd is a building block of most nixos modules. + + + From the project’s + webpage + + +systemd is a suite of basic building blocks for a Linux system. It provides a system and service manager that runs as PID 1 and starts the rest of the system. + +systemd provides aggressive parallelization capabilities, uses socket and D-Bus activation for starting services, offers on-demand starting of daemons, keeps track of processes using Linux control groups, maintains mount and automount points, and implements an elaborate transactional dependency-based service control logic. systemd supports SysV and LSB init scripts and works as a replacement for sysvinit. + +Other parts include a logging daemon, utilities to control basic system configuration like the hostname, date, locale, maintain a list of logged-in users and running containers and virtual machines, system accounts, runtime directories and settings, and daemons to manage simple network configuration, network time synchronization, log forwarding, and name resolution. + + + This chapter aims at describing the most commonly used features of + systemd in nixos. + + + Index + + + + + usage in modules + + + + + Starting a service + ref + + + + + Handling dependencies Talk about requires + wants after + partOf and the likes + ref + + + + + Environment Talk about environment how to + use it and not passing secrets + ref + + + + + Managing state Talk about StateDir RunDir WorkingDir and + when to use each + + + + + Initialisation of state and dependents (e.g. database) Talk + about adding a systemd service to handle database + initialisation + + + + + Database management Talk about connection via a unix socket + for more performance and reduced secret management + + + + + Managing secrets Talk about LoadCredential and how to use it + ref + + + + + Security and principle of least privilege Talk about + DynamicUser and user management + ref + + + + + Template units Show an example and explain why you would use + them + + + + + Advanced topics + + + + + Capabilities + ref + + + + + Socket activation for zero downtime deploys + + + + + + + + + + + +