From 48459567ae3e532a87267e186170eb931d7156a3 Mon Sep 17 00:00:00 2001 From: Maximilian Bosch Date: Wed, 8 Nov 2023 12:50:09 +0100 Subject: [PATCH 1/8] nixos/postgresql: drop ensurePermissions, fix ensureUsers for postgresql15 Closes #216989 First of all, a bit of context: in PostgreSQL, newly created users don't have the CREATE privilege on the public schema of a database even with `ALL PRIVILEGES` granted via `ensurePermissions` which is how most of the DB users are currently set up "declaratively"[1]. This means e.g. a freshly deployed Nextcloud service will break early because Nextcloud itself cannot CREATE any tables in the public schema anymore. The other issue here is that `ensurePermissions` is a mere hack. It's effectively a mixture of SQL code (e.g. `DATABASE foo` is relying on how a value is substituted in a query. You'd have to parse a subset of SQL to actually know which object are permissions granted to for a user). After analyzing the existing modules I realized that in every case with a single exception[2] the UNIX system user is equal to the db user is equal to the db name and I don't see a compelling reason why people would change that in 99% of the cases. In fact, some modules would even break if you'd change that because the declarations of the system user & the db user are mixed up[3]. So I decided to go with something new which restricts the ways to use `ensure*` options rather than expanding those[4]. Effectively this means that * The DB user _must_ be equal to the DB name. * Permissions are granted via `ensureDBOwnerhip` for an attribute-set in `ensureUsers`. That way, the user is actually the owner and can perform `CREATE`. * For such a postgres user, a database must be declared in `ensureDatabases`. For anything else, a custom state management should be implemented. This can either be `initialScript`, doing it manual, outside of the module or by implementing proper state management for postgresql[5], but the current state of `ensure*` isn't even declarative, but a convergent tool which is what Nix actually claims to _not_ do. Regarding existing setups: there are effectively two options: * Leave everything as-is (assuming that system user == db user == db name): then the DB user will automatically become the DB owner and everything else stays the same. * Drop the `createDatabase = true;` declarations: nothing will change because a removal of `ensure*` statements is ignored, so it doesn't matter at all whether this option is kept after the first deploy (and later on you'd usually restore from backups anyways). The DB user isn't the owner of the DB then, but for an existing setup this is irrelevant because CREATE on the public schema isn't revoked from existing users (only not granted for new users). [1] not really declarative though because removals of these statements are simply ignored for instance: https://github.com/NixOS/nixpkgs/issues/206467 [2] `services.invidious`: I removed the `ensure*` part temporarily because it IMHO falls into the category "manage the state on your own" (see the commit message). See also https://github.com/NixOS/nixpkgs/pull/265857 [3] e.g. roundcube had `"DATABASE ${cfg.database.username}" = "ALL PRIVILEGES";` [4] As opposed to other changes that are considered a potential fix, but also add more things like collation for DBs or passwords that are _never_ touched again when changing those. [5] As suggested in e.g. https://github.com/NixOS/nixpkgs/issues/206467 --- .../modules/services/databases/postgresql.nix | 65 +++++++++---------- nixos/modules/services/development/zammad.nix | 4 +- nixos/modules/services/finance/odoo.nix | 2 +- nixos/modules/services/mail/listmonk.nix | 2 +- nixos/modules/services/mail/roundcube.nix | 14 +++- nixos/modules/services/mail/sympa.nix | 10 ++- .../services/matrix/matrix-sliding-sync.nix | 4 +- .../services/matrix/mautrix-facebook.nix | 4 +- nixos/modules/services/misc/atuin.nix | 4 +- nixos/modules/services/misc/forgejo.nix | 10 ++- nixos/modules/services/misc/gitea.nix | 10 ++- nixos/modules/services/misc/redmine.nix | 4 +- .../services/misc/sourcehut/service.nix | 11 +++- .../services/monitoring/zabbix-proxy.nix | 4 +- .../services/monitoring/zabbix-server.nix | 4 +- .../modules/services/security/hockeypuck.nix | 2 +- nixos/modules/services/web-apps/coder.nix | 10 ++- .../modules/services/web-apps/gotosocial.nix | 4 +- nixos/modules/services/web-apps/invidious.nix | 6 -- nixos/modules/services/web-apps/lemmy.nix | 2 +- nixos/modules/services/web-apps/mastodon.nix | 6 +- nixos/modules/services/web-apps/mediawiki.nix | 4 +- nixos/modules/services/web-apps/miniflux.nix | 17 ++--- nixos/modules/services/web-apps/mobilizon.nix | 10 ++- nixos/modules/services/web-apps/moodle.nix | 4 +- nixos/modules/services/web-apps/netbox.nix | 4 +- nixos/modules/services/web-apps/nextcloud.nix | 2 +- .../modules/services/web-apps/onlyoffice.nix | 2 +- nixos/modules/services/web-apps/outline.nix | 2 +- .../services/web-apps/peering-manager.nix | 4 +- nixos/modules/services/web-apps/pixelfed.nix | 1 - nixos/modules/services/web-apps/tt-rss.nix | 13 +++- nixos/modules/services/web-servers/hydron.nix | 2 +- nixos/tests/dex-oidc.nix | 2 +- nixos/tests/ferretdb.nix | 2 +- nixos/tests/freshrss-pgsql.nix | 4 +- nixos/tests/grafana/basic.nix | 2 +- nixos/tests/hockeypuck.nix | 2 +- nixos/tests/home-assistant.nix | 12 ++-- nixos/tests/paperless.nix | 2 +- nixos/tests/pgadmin4.nix | 8 --- nixos/tests/pgbouncer.nix | 12 ++-- nixos/tests/powerdns-admin.nix | 4 +- nixos/tests/sftpgo.nix | 2 +- nixos/tests/tandoor-recipes.nix | 23 +++++++ nixos/tests/vikunja.nix | 2 +- nixos/tests/wiki-js.nix | 5 +- 47 files changed, 176 insertions(+), 153 deletions(-) diff --git a/nixos/modules/services/databases/postgresql.nix b/nixos/modules/services/databases/postgresql.nix index af4db5c9611f6..3d2205b63555f 100644 --- a/nixos/modules/services/databases/postgresql.nix +++ b/nixos/modules/services/databases/postgresql.nix @@ -165,25 +165,13 @@ in ''; }; - ensurePermissions = mkOption { - type = types.attrsOf types.str; - default = {}; - description = lib.mdDoc '' - Permissions to ensure for the user, specified as an attribute set. - The attribute names specify the database and tables to grant the permissions for. - The attribute values specify the permissions to grant. You may specify one or - multiple comma-separated SQL privileges here. - - For more information on how to specify the target - and on which privileges exist, see the - [GRANT syntax](https://www.postgresql.org/docs/current/sql-grant.html). - The attributes are used as `GRANT ''${attrValue} ON ''${attrName}`. - ''; - example = literalExpression '' - { - "DATABASE \"nextcloud\"" = "ALL PRIVILEGES"; - "ALL TABLES IN SCHEMA public" = "ALL PRIVILEGES"; - } + ensureDBOwnership = mkOption { + type = types.bool; + default = false; + description = mdDoc '' + Grants the user ownership to a database with the same name. + This database must be defined manually in + [](#opt-services.postgresql.ensureDatabases). ''; }; @@ -338,26 +326,21 @@ in }); default = []; description = lib.mdDoc '' - Ensures that the specified users exist and have at least the ensured permissions. + Ensures that the specified users exist. The PostgreSQL users will be identified using peer authentication. This authenticates the Unix user with the same name only, and that without the need for a password. - This option will never delete existing users or remove permissions, especially not when the value of this - option is changed. This means that users created and permissions assigned once through this option or - otherwise have to be removed manually. + This option will never delete existing users or remove DB ownership of databases + once granted with `ensureDBOwnership = true;`. This means that this must be + cleaned up manually when changing after changing the config in here. ''; example = literalExpression '' [ { name = "nextcloud"; - ensurePermissions = { - "DATABASE nextcloud" = "ALL PRIVILEGES"; - }; } { name = "superuser"; - ensurePermissions = { - "ALL TABLES IN SCHEMA public" = "ALL PRIVILEGES"; - }; + ensureDBOwnership = true; } ] ''; @@ -445,6 +428,19 @@ in config = mkIf cfg.enable { + assertions = [ + { + assertion = all + ({ name, ensureDBOwnership, ... }: ensureDBOwnership -> builtins.elem name cfg.ensureDatabases) + cfg.ensureUsers; + message = '' + For each database user defined with `services.postgresql.ensureUsers` and + `ensureDBOwnership = true;`, a database with the same name must be defined + in `services.postgresql.ensureDatabases`. + ''; + } + ]; + services.postgresql.settings = { hba_file = "${pkgs.writeText "pg_hba.conf" cfg.authentication}"; @@ -557,11 +553,9 @@ in concatMapStrings (user: let - userPermissions = concatStringsSep "\n" - (mapAttrsToList - (database: permission: ''$PSQL -tAc 'GRANT ${permission} ON ${database} TO "${user.name}"' '') - user.ensurePermissions - ); + dbOwnershipStmt = optionalString + user.ensureDBOwnership + ''$PSQL -tAc 'ALTER DATABASE "${user.name}" OWNER TO "${user.name}";' ''; filteredClauses = filterAttrs (name: value: value != null) user.ensureClauses; @@ -570,8 +564,9 @@ in userClauses = ''$PSQL -tAc 'ALTER ROLE "${user.name}" ${concatStringsSep " " clauseSqlStatements}' ''; in '' $PSQL -tAc "SELECT 1 FROM pg_roles WHERE rolname='${user.name}'" | grep -q 1 || $PSQL -tAc 'CREATE USER "${user.name}"' - ${userPermissions} ${userClauses} + + ${dbOwnershipStmt} '' ) cfg.ensureUsers diff --git a/nixos/modules/services/development/zammad.nix b/nixos/modules/services/development/zammad.nix index 7dd143eebf12f..d24ed24ef3956 100644 --- a/nixos/modules/services/development/zammad.nix +++ b/nixos/modules/services/development/zammad.nix @@ -204,7 +204,7 @@ in assertions = [ { - assertion = cfg.database.createLocally -> cfg.database.user == "zammad"; + assertion = cfg.database.createLocally -> cfg.database.user == "zammad" && cfg.database.name == "zammad"; message = "services.zammad.database.user must be set to \"zammad\" if services.zammad.database.createLocally is set to true"; } { @@ -231,7 +231,7 @@ in ensureUsers = [ { name = cfg.database.user; - ensurePermissions = { "DATABASE ${cfg.database.name}" = "ALL PRIVILEGES"; }; + ensureDBOwnership = true; } ]; }; diff --git a/nixos/modules/services/finance/odoo.nix b/nixos/modules/services/finance/odoo.nix index eec7c4e30cc48..b8574ed09af9c 100644 --- a/nixos/modules/services/finance/odoo.nix +++ b/nixos/modules/services/finance/odoo.nix @@ -121,7 +121,7 @@ in ensureDatabases = [ "odoo" ]; ensureUsers = [{ name = "odoo"; - ensurePermissions = { "DATABASE odoo" = "ALL PRIVILEGES"; }; + ensureDBOwnership = true; }]; }; }); diff --git a/nixos/modules/services/mail/listmonk.nix b/nixos/modules/services/mail/listmonk.nix index 11b2a5186229b..cea1bc9560815 100644 --- a/nixos/modules/services/mail/listmonk.nix +++ b/nixos/modules/services/mail/listmonk.nix @@ -168,7 +168,7 @@ in { ensureUsers = [{ name = "listmonk"; - ensurePermissions = { "DATABASE listmonk" = "ALL PRIVILEGES"; }; + ensureDBOwnership = true; }]; ensureDatabases = [ "listmonk" ]; diff --git a/nixos/modules/services/mail/roundcube.nix b/nixos/modules/services/mail/roundcube.nix index 22a4e3c451ab1..4e29f567ed92c 100644 --- a/nixos/modules/services/mail/roundcube.nix +++ b/nixos/modules/services/mail/roundcube.nix @@ -179,14 +179,22 @@ in }; }; + assertions = [ + { + assertion = localDB -> cfg.database.username == cfg.database.dbname; + message = '' + When setting up a DB and its owner user, the owner and the DB name must be + equal! + ''; + } + ]; + services.postgresql = mkIf localDB { enable = true; ensureDatabases = [ cfg.database.dbname ]; ensureUsers = [ { name = cfg.database.username; - ensurePermissions = { - "DATABASE ${cfg.database.username}" = "ALL PRIVILEGES"; - }; + ensureDBOwnership = true; } ]; }; diff --git a/nixos/modules/services/mail/sympa.nix b/nixos/modules/services/mail/sympa.nix index 7a5047b2bea54..04ae46f66eeaf 100644 --- a/nixos/modules/services/mail/sympa.nix +++ b/nixos/modules/services/mail/sympa.nix @@ -218,7 +218,7 @@ in default = null; example = "/run/keys/sympa-dbpassword"; description = lib.mdDoc '' - A file containing the password for {option}`services.sympa.database.user`. + A file containing the password for {option}`services.sympa.database.name`. ''; }; @@ -342,6 +342,7 @@ in db_type = cfg.database.type; db_name = cfg.database.name; + db_user = cfg.database.name; } // (optionalAttrs (cfg.database.host != null) { db_host = cfg.database.host; @@ -355,9 +356,6 @@ in // (optionalAttrs (cfg.database.port != null) { db_port = cfg.database.port; }) - // (optionalAttrs (cfg.database.user != null) { - db_user = cfg.database.user; - }) // (optionalAttrs (cfg.mta.type == "postfix") { sendmail_aliases = "${dataDir}/sympa_transport"; aliases_program = "${pkgs.postfix}/bin/postmap"; @@ -393,7 +391,7 @@ in users.groups.${group} = {}; assertions = [ - { assertion = cfg.database.createLocally -> cfg.database.user == user; + { assertion = cfg.database.createLocally -> cfg.database.user == user && cfg.database.name == cfg.database.user; message = "services.sympa.database.user must be set to ${user} if services.sympa.database.createLocally is set to true"; } { assertion = cfg.database.createLocally -> cfg.database.passwordFile == null; @@ -579,7 +577,7 @@ in ensureDatabases = [ cfg.database.name ]; ensureUsers = [ { name = cfg.database.user; - ensurePermissions = { "DATABASE ${cfg.database.name}" = "ALL PRIVILEGES"; }; + ensureDBOwnership = true; } ]; }; diff --git a/nixos/modules/services/matrix/matrix-sliding-sync.nix b/nixos/modules/services/matrix/matrix-sliding-sync.nix index 9807cde409197..84bb38f35aeb0 100644 --- a/nixos/modules/services/matrix/matrix-sliding-sync.nix +++ b/nixos/modules/services/matrix/matrix-sliding-sync.nix @@ -74,9 +74,9 @@ in services.postgresql = lib.optionalAttrs cfg.createDatabase { enable = true; ensureDatabases = [ "matrix-sliding-sync" ]; - ensureUsers = [ rec { + ensureUsers = [ { name = "matrix-sliding-sync"; - ensurePermissions."DATABASE \"${name}\"" = "ALL PRIVILEGES"; + ensureDBOwnership = true; } ]; }; diff --git a/nixos/modules/services/matrix/mautrix-facebook.nix b/nixos/modules/services/matrix/mautrix-facebook.nix index 671040500df84..d7cf024bb8077 100644 --- a/nixos/modules/services/matrix/mautrix-facebook.nix +++ b/nixos/modules/services/matrix/mautrix-facebook.nix @@ -135,9 +135,7 @@ in { ensureDatabases = ["mautrix-facebook"]; ensureUsers = [{ name = "mautrix-facebook"; - ensurePermissions = { - "DATABASE \"mautrix-facebook\"" = "ALL PRIVILEGES"; - }; + ensureDBOwnership = true; }]; }; diff --git a/nixos/modules/services/misc/atuin.nix b/nixos/modules/services/misc/atuin.nix index 8d2c1b5242ffa..2d6ffc510ce55 100644 --- a/nixos/modules/services/misc/atuin.nix +++ b/nixos/modules/services/misc/atuin.nix @@ -73,9 +73,7 @@ in enable = true; ensureUsers = [{ name = "atuin"; - ensurePermissions = { - "DATABASE atuin" = "ALL PRIVILEGES"; - }; + ensureDBOwnership = true; }]; ensureDatabases = [ "atuin" ]; }; diff --git a/nixos/modules/services/misc/forgejo.nix b/nixos/modules/services/misc/forgejo.nix index 90b5f16f4189b..6f459048f347e 100644 --- a/nixos/modules/services/misc/forgejo.nix +++ b/nixos/modules/services/misc/forgejo.nix @@ -357,6 +357,14 @@ in assertion = cfg.database.createDatabase -> useSqlite || cfg.database.user == cfg.user; message = "services.forgejo.database.user must match services.forgejo.user if the database is to be automatically provisioned"; } + { assertion = cfg.database.createDatabase && usePostgresql -> cfg.database.user == cfg.database.name; + message = '' + When creating a database via NixOS, the db user and db name must be equal! + If you already have an existing DB+user and this assertion is new, you can safely set + `services.forgejo.createDatabase` to `false` because removal of `ensureUsers` + and `ensureDatabases` doesn't have any effect. + ''; + } ]; services.forgejo.settings = { @@ -423,7 +431,7 @@ in ensureUsers = [ { name = cfg.database.user; - ensurePermissions = { "DATABASE ${cfg.database.name}" = "ALL PRIVILEGES"; }; + ensureDBOwnership = true; } ]; }; diff --git a/nixos/modules/services/misc/gitea.nix b/nixos/modules/services/misc/gitea.nix index 3f690f85d623b..be528a2989918 100644 --- a/nixos/modules/services/misc/gitea.nix +++ b/nixos/modules/services/misc/gitea.nix @@ -394,6 +394,14 @@ in { assertion = cfg.database.createDatabase -> useSqlite || cfg.database.user == cfg.user; message = "services.gitea.database.user must match services.gitea.user if the database is to be automatically provisioned"; } + { assertion = cfg.database.createDatabase && usePostgresql -> cfg.database.user == cfg.database.name; + message = '' + When creating a database via NixOS, the db user and db name must be equal! + If you already have an existing DB+user and this assertion is new, you can safely set + `services.gitea.createDatabase` to `false` because removal of `ensureUsers` + and `ensureDatabases` doesn't have any effect. + ''; + } ]; services.gitea.settings = { @@ -461,7 +469,7 @@ in ensureDatabases = [ cfg.database.name ]; ensureUsers = [ { name = cfg.database.user; - ensurePermissions = { "DATABASE ${cfg.database.name}" = "ALL PRIVILEGES"; }; + ensureDBOwnership = true; } ]; }; diff --git a/nixos/modules/services/misc/redmine.nix b/nixos/modules/services/misc/redmine.nix index a296fd3816bbb..20fa71507b6b0 100644 --- a/nixos/modules/services/misc/redmine.nix +++ b/nixos/modules/services/misc/redmine.nix @@ -267,7 +267,7 @@ in { assertion = cfg.database.passwordFile != null || cfg.database.socket != null; message = "one of services.redmine.database.socket or services.redmine.database.passwordFile must be set"; } - { assertion = cfg.database.createLocally -> cfg.database.user == cfg.user; + { assertion = cfg.database.createLocally -> cfg.database.user == cfg.user && cfg.database.user == cfg.database.name; message = "services.redmine.database.user must be set to ${cfg.user} if services.redmine.database.createLocally is set true"; } { assertion = cfg.database.createLocally -> cfg.database.socket != null; @@ -315,7 +315,7 @@ in ensureDatabases = [ cfg.database.name ]; ensureUsers = [ { name = cfg.database.user; - ensurePermissions = { "DATABASE ${cfg.database.name}" = "ALL PRIVILEGES"; }; + ensureDBOwnership = true; } ]; }; diff --git a/nixos/modules/services/misc/sourcehut/service.nix b/nixos/modules/services/misc/sourcehut/service.nix index 18c2f5effc5a5..d2cd599d3fec1 100644 --- a/nixos/modules/services/misc/sourcehut/service.nix +++ b/nixos/modules/services/misc/sourcehut/service.nix @@ -242,6 +242,15 @@ in } cfg.nginx.virtualHost ]; }; + assertions = [ + { + assertion = srvCfg.user == srvCfg.postgresql.database; + message = '' + When creating a database via NixOS, the db user and db name must be equal! + ''; + } + ]; + services.postgresql = mkIf cfg.postgresql.enable { authentication = '' local ${srvCfg.postgresql.database} ${srvCfg.user} trust @@ -249,7 +258,7 @@ in ensureDatabases = [ srvCfg.postgresql.database ]; ensureUsers = map (name: { inherit name; - ensurePermissions = { "DATABASE \"${srvCfg.postgresql.database}\"" = "ALL PRIVILEGES"; }; + ensureDBOwnership = true; }) [srvCfg.user]; }; diff --git a/nixos/modules/services/monitoring/zabbix-proxy.nix b/nixos/modules/services/monitoring/zabbix-proxy.nix index 85da416ba6c33..503e81b48a587 100644 --- a/nixos/modules/services/monitoring/zabbix-proxy.nix +++ b/nixos/modules/services/monitoring/zabbix-proxy.nix @@ -203,7 +203,7 @@ in { assertion = !config.services.zabbixServer.enable; message = "Please choose one of services.zabbixServer or services.zabbixProxy."; } - { assertion = cfg.database.createLocally -> cfg.database.user == user; + { assertion = cfg.database.createLocally -> cfg.database.user == user && cfg.database.name == cfg.database.user; message = "services.zabbixProxy.database.user must be set to ${user} if services.zabbixProxy.database.createLocally is set true"; } { assertion = cfg.database.createLocally -> cfg.database.passwordFile == null; @@ -252,7 +252,7 @@ in ensureDatabases = [ cfg.database.name ]; ensureUsers = [ { name = cfg.database.user; - ensurePermissions = { "DATABASE ${cfg.database.name}" = "ALL PRIVILEGES"; }; + ensureDBOwnership = true; } ]; }; diff --git a/nixos/modules/services/monitoring/zabbix-server.nix b/nixos/modules/services/monitoring/zabbix-server.nix index 2b50280e3969a..0607188d21319 100644 --- a/nixos/modules/services/monitoring/zabbix-server.nix +++ b/nixos/modules/services/monitoring/zabbix-server.nix @@ -191,7 +191,7 @@ in config = mkIf cfg.enable { assertions = [ - { assertion = cfg.database.createLocally -> cfg.database.user == user; + { assertion = cfg.database.createLocally -> cfg.database.user == user && cfg.database.user == cfg.database.name; message = "services.zabbixServer.database.user must be set to ${user} if services.zabbixServer.database.createLocally is set true"; } { assertion = cfg.database.createLocally -> cfg.database.passwordFile == null; @@ -240,7 +240,7 @@ in ensureDatabases = [ cfg.database.name ]; ensureUsers = [ { name = cfg.database.user; - ensurePermissions = { "DATABASE ${cfg.database.name}" = "ALL PRIVILEGES"; }; + ensureDBOwnership = true; } ]; }; diff --git a/nixos/modules/services/security/hockeypuck.nix b/nixos/modules/services/security/hockeypuck.nix index 127134bc5dba5..56c13d791920c 100644 --- a/nixos/modules/services/security/hockeypuck.nix +++ b/nixos/modules/services/security/hockeypuck.nix @@ -55,7 +55,7 @@ in { ensureDatabases = [ "hockeypuck" ]; ensureUsers = [{ name = "hockeypuck"; - ensurePermissions."DATABASE hockeypuck" = "ALL PRIVILEGES"; + ensureDBOwnership = true; }]; }; ``` diff --git a/nixos/modules/services/web-apps/coder.nix b/nixos/modules/services/web-apps/coder.nix index 469a29bc3aa8c..f65211308c402 100644 --- a/nixos/modules/services/web-apps/coder.nix +++ b/nixos/modules/services/web-apps/coder.nix @@ -149,8 +149,8 @@ in { config = mkIf cfg.enable { assertions = [ - { assertion = cfg.database.createLocally -> cfg.database.username == name; - message = "services.coder.database.username must be set to ${user} if services.coder.database.createLocally is set true"; + { assertion = cfg.database.createLocally -> cfg.database.username == name && cfg.database.database == cfg.database.username; + message = "services.coder.database.username must be set to ${name} if services.coder.database.createLocally is set true"; } ]; @@ -193,10 +193,8 @@ in { cfg.database.database ]; ensureUsers = [{ - name = cfg.database.username; - ensurePermissions = { - "DATABASE \"${cfg.database.database}\"" = "ALL PRIVILEGES"; - }; + name = cfg.user; + ensureDBOwnership = true; } ]; }; diff --git a/nixos/modules/services/web-apps/gotosocial.nix b/nixos/modules/services/web-apps/gotosocial.nix index f7ae018d5b7cc..9c21719a57590 100644 --- a/nixos/modules/services/web-apps/gotosocial.nix +++ b/nixos/modules/services/web-apps/gotosocial.nix @@ -128,9 +128,7 @@ in ensureUsers = [ { name = "gotosocial"; - ensurePermissions = { - "DATABASE gotosocial" = "ALL PRIVILEGES"; - }; + ensureDBOwnership = true; } ]; }; diff --git a/nixos/modules/services/web-apps/invidious.nix b/nixos/modules/services/web-apps/invidious.nix index 5603ef7392e82..fc9c1ec06f654 100644 --- a/nixos/modules/services/web-apps/invidious.nix +++ b/nixos/modules/services/web-apps/invidious.nix @@ -112,12 +112,6 @@ let services.postgresql = { enable = true; ensureDatabases = lib.singleton cfg.settings.db.dbname; - ensureUsers = lib.singleton { - name = cfg.settings.db.user; - ensurePermissions = { - "DATABASE ${cfg.settings.db.dbname}" = "ALL PRIVILEGES"; - }; - }; # This is only needed because the unix user invidious isn't the same as # the database user. This tells postgres to map one to the other. identMap = '' diff --git a/nixos/modules/services/web-apps/lemmy.nix b/nixos/modules/services/web-apps/lemmy.nix index 20d9dcb7c2661..32389f7a59dd8 100644 --- a/nixos/modules/services/web-apps/lemmy.nix +++ b/nixos/modules/services/web-apps/lemmy.nix @@ -146,7 +146,7 @@ in ensureDatabases = [ cfg.settings.database.database ]; ensureUsers = [{ name = cfg.settings.database.user; - ensurePermissions."DATABASE ${cfg.settings.database.database}" = "ALL PRIVILEGES"; + ensureDBOwnership = true; }]; }; diff --git a/nixos/modules/services/web-apps/mastodon.nix b/nixos/modules/services/web-apps/mastodon.nix index 2aab97438b7d6..5feca2525ea15 100644 --- a/nixos/modules/services/web-apps/mastodon.nix +++ b/nixos/modules/services/web-apps/mastodon.nix @@ -557,7 +557,7 @@ in { config = lib.mkIf cfg.enable (lib.mkMerge [{ assertions = [ { - assertion = databaseActuallyCreateLocally -> (cfg.user == cfg.database.user); + assertion = databaseActuallyCreateLocally -> (cfg.user == cfg.database.user && cfg.database.user == cfg.database.name); message = '' For local automatic database provisioning (services.mastodon.database.createLocally == true) with peer authentication (services.mastodon.database.host == "/run/postgresql") to work services.mastodon.user @@ -799,8 +799,8 @@ in { enable = true; ensureUsers = [ { - name = cfg.database.user; - ensurePermissions."DATABASE ${cfg.database.name}" = "ALL PRIVILEGES"; + name = cfg.database.name; + ensureDBOwnership = true; } ]; ensureDatabases = [ cfg.database.name ]; diff --git a/nixos/modules/services/web-apps/mediawiki.nix b/nixos/modules/services/web-apps/mediawiki.nix index 8b494b7c12083..ce7bcd94b3f01 100644 --- a/nixos/modules/services/web-apps/mediawiki.nix +++ b/nixos/modules/services/web-apps/mediawiki.nix @@ -454,7 +454,7 @@ in { assertion = cfg.database.createLocally -> (cfg.database.type == "mysql" || cfg.database.type == "postgres"); message = "services.mediawiki.createLocally is currently only supported for database type 'mysql' and 'postgres'"; } - { assertion = cfg.database.createLocally -> cfg.database.user == user; + { assertion = cfg.database.createLocally -> cfg.database.user == user && cfg.database.name == cfg.database.user; message = "services.mediawiki.database.user must be set to ${user} if services.mediawiki.database.createLocally is set true"; } { assertion = cfg.database.createLocally -> cfg.database.socket != null; @@ -486,7 +486,7 @@ in ensureDatabases = [ cfg.database.name ]; ensureUsers = [{ name = cfg.database.user; - ensurePermissions = { "DATABASE \"${cfg.database.name}\"" = "ALL PRIVILEGES"; }; + ensureDBOwnership = true; }]; }; diff --git a/nixos/modules/services/web-apps/miniflux.nix b/nixos/modules/services/web-apps/miniflux.nix index 3374c746ad3dd..5c8c93c13c434 100644 --- a/nixos/modules/services/web-apps/miniflux.nix +++ b/nixos/modules/services/web-apps/miniflux.nix @@ -6,13 +6,10 @@ let defaultAddress = "localhost:8080"; - dbUser = "miniflux"; - dbName = "miniflux"; - pgbin = "${config.services.postgresql.package}/bin"; preStart = pkgs.writeScript "miniflux-pre-start" '' #!${pkgs.runtimeShell} - ${pgbin}/psql "${dbName}" -c "CREATE EXTENSION IF NOT EXISTS hstore" + ${pgbin}/psql "miniflux" -c "CREATE EXTENSION IF NOT EXISTS hstore" ''; in @@ -62,7 +59,7 @@ in services.miniflux.config = { LISTEN_ADDR = mkDefault defaultAddress; - DATABASE_URL = "user=${dbUser} host=/run/postgresql dbname=${dbName}"; + DATABASE_URL = "user=miniflux host=/run/postgresql dbname=miniflux"; RUN_MIGRATIONS = "1"; CREATE_ADMIN = "1"; }; @@ -70,12 +67,10 @@ in services.postgresql = { enable = true; ensureUsers = [ { - name = dbUser; - ensurePermissions = { - "DATABASE ${dbName}" = "ALL PRIVILEGES"; - }; + name = "miniflux"; + ensureDBOwnership = true; } ]; - ensureDatabases = [ dbName ]; + ensureDatabases = [ "miniflux" ]; }; systemd.services.miniflux-dbsetup = { @@ -97,7 +92,7 @@ in serviceConfig = { ExecStart = "${cfg.package}/bin/miniflux"; - User = dbUser; + User = "miniflux"; DynamicUser = true; RuntimeDirectory = "miniflux"; RuntimeDirectoryMode = "0700"; diff --git a/nixos/modules/services/web-apps/mobilizon.nix b/nixos/modules/services/web-apps/mobilizon.nix index 343c5cead2b15..601c2830e0e27 100644 --- a/nixos/modules/services/web-apps/mobilizon.nix +++ b/nixos/modules/services/web-apps/mobilizon.nix @@ -212,6 +212,12 @@ in assertion = cfg.nginx.enable -> (cfg.settings.":mobilizon"."Mobilizon.Web.Endpoint".http.ip == settingsFormat.lib.mkTuple [ 0 0 0 0 0 0 0 1 ]); message = "Setting the IP mobilizon listens on is only possible when the nginx config is not used, as it is hardcoded there."; } + { + assertion = isLocalPostgres -> repoSettings.database == repoSettings.username; + message = '' + When creating a database via NixOS, the db user and db name must be equal! + ''; + } ]; services.mobilizon.settings = { @@ -372,9 +378,7 @@ in ensureUsers = [ { name = dbUser; - ensurePermissions = { - "DATABASE \"${repoSettings.database}\"" = "ALL PRIVILEGES"; - }; + ensureDBOwnership = true; } ]; extraPlugins = with postgresql.pkgs; [ postgis ]; diff --git a/nixos/modules/services/web-apps/moodle.nix b/nixos/modules/services/web-apps/moodle.nix index b617e9a593795..04ae6bd7f1754 100644 --- a/nixos/modules/services/web-apps/moodle.nix +++ b/nixos/modules/services/web-apps/moodle.nix @@ -194,7 +194,7 @@ in config = mkIf cfg.enable { assertions = [ - { assertion = cfg.database.createLocally -> cfg.database.user == user; + { assertion = cfg.database.createLocally -> cfg.database.user == user && cfg.database.user == cfg.database.name; message = "services.moodle.database.user must be set to ${user} if services.moodle.database.createLocally is set true"; } { assertion = cfg.database.createLocally -> cfg.database.passwordFile == null; @@ -220,7 +220,7 @@ in ensureDatabases = [ cfg.database.name ]; ensureUsers = [ { name = cfg.database.user; - ensurePermissions = { "DATABASE ${cfg.database.name}" = "ALL PRIVILEGES"; }; + ensureDBOwnership = true; } ]; }; diff --git a/nixos/modules/services/web-apps/netbox.nix b/nixos/modules/services/web-apps/netbox.nix index 8ba1852848e5b..3b9434e3d3456 100644 --- a/nixos/modules/services/web-apps/netbox.nix +++ b/nixos/modules/services/web-apps/netbox.nix @@ -257,9 +257,7 @@ in { ensureUsers = [ { name = "netbox"; - ensurePermissions = { - "DATABASE netbox" = "ALL PRIVILEGES"; - }; + ensureDBOwnership = true; } ]; }; diff --git a/nixos/modules/services/web-apps/nextcloud.nix b/nixos/modules/services/web-apps/nextcloud.nix index f9713cac47e94..f1ac3770d403c 100644 --- a/nixos/modules/services/web-apps/nextcloud.nix +++ b/nixos/modules/services/web-apps/nextcloud.nix @@ -1042,7 +1042,7 @@ in { ensureDatabases = [ cfg.config.dbname ]; ensureUsers = [{ name = cfg.config.dbuser; - ensurePermissions = { "DATABASE ${cfg.config.dbname}" = "ALL PRIVILEGES"; }; + ensureDBOwnership = true; }]; }; diff --git a/nixos/modules/services/web-apps/onlyoffice.nix b/nixos/modules/services/web-apps/onlyoffice.nix index 3494f2fa21f09..f958566b91f09 100644 --- a/nixos/modules/services/web-apps/onlyoffice.nix +++ b/nixos/modules/services/web-apps/onlyoffice.nix @@ -198,7 +198,7 @@ in ensureDatabases = [ "onlyoffice" ]; ensureUsers = [{ name = "onlyoffice"; - ensurePermissions = { "DATABASE \"onlyoffice\"" = "ALL PRIVILEGES"; }; + ensureDBOwnership = true; }]; }; }; diff --git a/nixos/modules/services/web-apps/outline.nix b/nixos/modules/services/web-apps/outline.nix index 0e3bd07c1fc14..d97b45d624187 100644 --- a/nixos/modules/services/web-apps/outline.nix +++ b/nixos/modules/services/web-apps/outline.nix @@ -581,7 +581,7 @@ in enable = true; ensureUsers = [{ name = "outline"; - ensurePermissions."DATABASE outline" = "ALL PRIVILEGES"; + ensureDBOwnership = true; }]; ensureDatabases = [ "outline" ]; }; diff --git a/nixos/modules/services/web-apps/peering-manager.nix b/nixos/modules/services/web-apps/peering-manager.nix index 7012df6dffbfd..d6f6077268d46 100644 --- a/nixos/modules/services/web-apps/peering-manager.nix +++ b/nixos/modules/services/web-apps/peering-manager.nix @@ -186,9 +186,7 @@ in { ensureUsers = [ { name = "peering-manager"; - ensurePermissions = { - "DATABASE \"peering-manager\"" = "ALL PRIVILEGES"; - }; + ensureDBOwnership = true; } ]; }; diff --git a/nixos/modules/services/web-apps/pixelfed.nix b/nixos/modules/services/web-apps/pixelfed.nix index 159fb52476aab..b0a25dcce9efe 100644 --- a/nixos/modules/services/web-apps/pixelfed.nix +++ b/nixos/modules/services/web-apps/pixelfed.nix @@ -271,7 +271,6 @@ in { ensureDatabases = [ cfg.database.name ]; ensureUsers = [{ name = user; - ensurePermissions = { }; }]; }; diff --git a/nixos/modules/services/web-apps/tt-rss.nix b/nixos/modules/services/web-apps/tt-rss.nix index 7b2e3be4295e8..a8fb37d2c5ecd 100644 --- a/nixos/modules/services/web-apps/tt-rss.nix +++ b/nixos/modules/services/web-apps/tt-rss.nix @@ -529,6 +529,15 @@ let assertion = cfg.database.password != null -> cfg.database.passwordFile == null; message = "Cannot set both password and passwordFile"; } + { + assertion = cfg.database.createLocally -> cfg.database.name == cfg.user && cfg.database.user == cfg.user; + message = '' + When creating a database via NixOS, the db user and db name must be equal! + If you already have an existing DB+user and this assertion is new, you can safely set + `services.tt-rss.database.createLocally` to `false` because removal of `ensureUsers` + and `ensureDatabases` doesn't have any effect. + ''; + } ]; services.phpfpm.pools = mkIf (cfg.pool == "${poolName}") { @@ -632,8 +641,8 @@ let enable = mkDefault true; ensureDatabases = [ cfg.database.name ]; ensureUsers = [ - { name = cfg.user; - ensurePermissions = { "DATABASE ${cfg.database.name}" = "ALL PRIVILEGES"; }; + { name = cfg.database.user; + ensureDBOwnership = true; } ]; }; diff --git a/nixos/modules/services/web-servers/hydron.nix b/nixos/modules/services/web-servers/hydron.nix index 4434965b217a0..9d30fdc0caab0 100644 --- a/nixos/modules/services/web-servers/hydron.nix +++ b/nixos/modules/services/web-servers/hydron.nix @@ -93,7 +93,7 @@ in with lib; { ensureDatabases = [ "hydron" ]; ensureUsers = [ { name = "hydron"; - ensurePermissions = { "DATABASE hydron" = "ALL PRIVILEGES"; }; + ensureDBOwnership = true; } ]; }; diff --git a/nixos/tests/dex-oidc.nix b/nixos/tests/dex-oidc.nix index 37275a97ef0fb..e54ae18ca9373 100644 --- a/nixos/tests/dex-oidc.nix +++ b/nixos/tests/dex-oidc.nix @@ -49,7 +49,7 @@ import ./make-test-python.nix ({ lib, ... }: { ensureUsers = [ { name = "dex"; - ensurePermissions = { "DATABASE dex" = "ALL PRIVILEGES"; }; + ensureDBOwnership = true; } ]; }; diff --git a/nixos/tests/ferretdb.nix b/nixos/tests/ferretdb.nix index 9ad7397ade80c..7251198af77dc 100644 --- a/nixos/tests/ferretdb.nix +++ b/nixos/tests/ferretdb.nix @@ -39,7 +39,7 @@ with import ../lib/testing-python.nix { inherit system; }; ensureDatabases = [ "ferretdb" ]; ensureUsers = [{ name = "ferretdb"; - ensurePermissions."DATABASE ferretdb" = "ALL PRIVILEGES"; + ensureDBOwnership = true; }]; }; diff --git a/nixos/tests/freshrss-pgsql.nix b/nixos/tests/freshrss-pgsql.nix index 055bd51ed43d7..c685f4a8159b8 100644 --- a/nixos/tests/freshrss-pgsql.nix +++ b/nixos/tests/freshrss-pgsql.nix @@ -22,9 +22,7 @@ import ./make-test-python.nix ({ lib, pkgs, ... }: { ensureUsers = [ { name = "freshrss"; - ensurePermissions = { - "DATABASE freshrss" = "ALL PRIVILEGES"; - }; + ensureDBOwnership = true; } ]; initialScript = pkgs.writeText "postgresql-password" '' diff --git a/nixos/tests/grafana/basic.nix b/nixos/tests/grafana/basic.nix index 8bf4caad7fbfc..dd389bc8a3d1f 100644 --- a/nixos/tests/grafana/basic.nix +++ b/nixos/tests/grafana/basic.nix @@ -55,7 +55,7 @@ let ensureDatabases = [ "grafana" ]; ensureUsers = [{ name = "grafana"; - ensurePermissions."DATABASE grafana" = "ALL PRIVILEGES"; + ensureDBOwnership = true; }]; }; systemd.services.grafana.after = [ "postgresql.service" ]; diff --git a/nixos/tests/hockeypuck.nix b/nixos/tests/hockeypuck.nix index 2b9dba8720aba..675d6b226ad20 100644 --- a/nixos/tests/hockeypuck.nix +++ b/nixos/tests/hockeypuck.nix @@ -35,7 +35,7 @@ in { ensureDatabases = [ "hockeypuck" ]; ensureUsers = [{ name = "hockeypuck"; - ensurePermissions."DATABASE hockeypuck" = "ALL PRIVILEGES"; + ensureDBOwnership = true; }]; }; }; diff --git a/nixos/tests/home-assistant.nix b/nixos/tests/home-assistant.nix index e97e8a467b187..e1588088ba198 100644 --- a/nixos/tests/home-assistant.nix +++ b/nixos/tests/home-assistant.nix @@ -9,13 +9,11 @@ in { nodes.hass = { pkgs, ... }: { services.postgresql = { enable = true; - - # FIXME: hack for https://github.com/NixOS/nixpkgs/issues/216989 - # Should be replaced with ensureUsers again when a solution for that is found - initialScript = pkgs.writeText "hass-setup-db.sql" '' - CREATE ROLE hass WITH LOGIN; - CREATE DATABASE hass WITH OWNER hass; - ''; + ensureDatabases = [ "hass" ]; + ensureUsers = [{ + name = "hass"; + ensureDBOwnership = true; + }]; }; services.home-assistant = { diff --git a/nixos/tests/paperless.nix b/nixos/tests/paperless.nix index 22409e8992367..6a51cc522bdc5 100644 --- a/nixos/tests/paperless.nix +++ b/nixos/tests/paperless.nix @@ -17,7 +17,7 @@ import ./make-test-python.nix ({ lib, ... }: { ensureDatabases = [ "paperless" ]; ensureUsers = [ { name = config.services.paperless.user; - ensurePermissions = { "DATABASE \"paperless\"" = "ALL PRIVILEGES"; }; + ensureDBOwnership = true; } ]; }; diff --git a/nixos/tests/pgadmin4.nix b/nixos/tests/pgadmin4.nix index cb8de87c9ee31..3ee7ed19fa1c5 100644 --- a/nixos/tests/pgadmin4.nix +++ b/nixos/tests/pgadmin4.nix @@ -19,14 +19,6 @@ import ./make-test-python.nix ({ pkgs, lib, ... }: authentication = '' host all all localhost trust ''; - ensureUsers = [ - { - name = "postgres"; - ensurePermissions = { - "DATABASE \"postgres\"" = "ALL PRIVILEGES"; - }; - } - ]; }; services.pgadmin = { diff --git a/nixos/tests/pgbouncer.nix b/nixos/tests/pgbouncer.nix index 1e72327d42005..814ca0d58865e 100644 --- a/nixos/tests/pgbouncer.nix +++ b/nixos/tests/pgbouncer.nix @@ -24,13 +24,11 @@ in services = { postgresql = { enable = true; - ensureDatabases = [ "testdb" ]; + ensureDatabases = [ "test" ]; ensureUsers = [ { - name = "testuser"; - ensurePermissions = { - "DATABASE testdb" = "ALL PRIVILEGES"; - }; + name = "test"; + ensureDBOwnership = true; }]; authentication = '' local testdb testuser scram-sha-256 @@ -40,7 +38,7 @@ in pgbouncer = { enable = true; listenAddress = "localhost"; - databases = { testdb = "host=/run/postgresql/ port=5432 auth_user=testuser dbname=testdb"; }; + databases = { test = "host=/run/postgresql/ port=5432 auth_user=testuser dbname=test"; }; authType = "scram-sha-256"; authFile = testAuthFile; }; @@ -55,7 +53,7 @@ in # Test if we can make a query through PgBouncer one.wait_until_succeeds( - "psql 'postgres://testuser:testpass@localhost:6432/testdb' -c 'SELECT 1;'" + "psql 'postgres://testuser:testpass@localhost:6432/test' -c 'SELECT 1;'" ) ''; }) diff --git a/nixos/tests/powerdns-admin.nix b/nixos/tests/powerdns-admin.nix index d7bacb24eec5d..d326d74a98261 100644 --- a/nixos/tests/powerdns-admin.nix +++ b/nixos/tests/powerdns-admin.nix @@ -87,9 +87,7 @@ let ensureUsers = [ { name = "powerdnsadmin"; - ensurePermissions = { - "DATABASE powerdnsadmin" = "ALL PRIVILEGES"; - }; + ensureDBOwnership = true; } ]; }; diff --git a/nixos/tests/sftpgo.nix b/nixos/tests/sftpgo.nix index db0098d2ac48c..a5bb1981d2c3c 100644 --- a/nixos/tests/sftpgo.nix +++ b/nixos/tests/sftpgo.nix @@ -156,7 +156,7 @@ in ensureDatabases = [ "sftpgo" ]; ensureUsers = [{ name = "sftpgo"; - ensurePermissions."DATABASE sftpgo" = "ALL PRIVILEGES"; + ensureDBOwnership = true; }]; }; diff --git a/nixos/tests/tandoor-recipes.nix b/nixos/tests/tandoor-recipes.nix index f3369da99a055..18beaac6f062c 100644 --- a/nixos/tests/tandoor-recipes.nix +++ b/nixos/tests/tandoor-recipes.nix @@ -5,6 +5,29 @@ import ./make-test-python.nix ({ lib, ... }: { nodes.machine = { pkgs, ... }: { services.tandoor-recipes = { enable = true; + extraConfig = { + DB_ENGINE = "django.db.backends.postgresql"; + POSTGRES_HOST = "/run/postgresql"; + POSTGRES_USER = "tandoor_recipes"; + POSTGRES_DB = "tandoor_recipes"; + }; + }; + + services.postgresql = { + enable = true; + ensureDatabases = [ "tandoor_recipes" ]; + ensureUsers = [ + { + name = "tandoor_recipes"; + ensureDBOwnership = true; + } + ]; + }; + + systemd.services = { + tandoor-recipes = { + after = [ "postgresql.service" ]; + }; }; }; diff --git a/nixos/tests/vikunja.nix b/nixos/tests/vikunja.nix index 2660aa9767ca7..60fd5ce13854e 100644 --- a/nixos/tests/vikunja.nix +++ b/nixos/tests/vikunja.nix @@ -33,7 +33,7 @@ import ./make-test-python.nix ({ pkgs, lib, ... }: { ensureDatabases = [ "vikunja-api" ]; ensureUsers = [ { name = "vikunja-api"; - ensurePermissions = { "DATABASE \"vikunja-api\"" = "ALL PRIVILEGES"; }; + ensureDBOwnership = true; } ]; }; diff --git a/nixos/tests/wiki-js.nix b/nixos/tests/wiki-js.nix index fd054a9c59093..8b3c51935a6c4 100644 --- a/nixos/tests/wiki-js.nix +++ b/nixos/tests/wiki-js.nix @@ -10,14 +10,15 @@ import ./make-test-python.nix ({ pkgs, lib, ...} : { enable = true; settings.db.host = "/run/postgresql"; settings.db.user = "wiki-js"; + settings.db.db = "wiki-js"; settings.logLevel = "debug"; }; services.postgresql = { enable = true; - ensureDatabases = [ "wiki" ]; + ensureDatabases = [ "wiki-js" ]; ensureUsers = [ { name = "wiki-js"; - ensurePermissions."DATABASE wiki" = "ALL PRIVILEGES"; + ensureDBOwnership = true; } ]; }; From 12797a6a39393c61b00d6e0085eeaffbb1ba1d3c Mon Sep 17 00:00:00 2001 From: Raito Bezarius Date: Mon, 13 Nov 2023 14:55:16 +0100 Subject: [PATCH 2/8] nixos/postgresql: restore `ensurePermissions` and strong-deprecate it As it is technically a breaking change, we should at least make a strong deprecation of `ensurePermissions` and leave it in the broken state it is, for out of tree users. We give them a 6 months notice to migrate away by doing so, which is honest. In the meantime, we forbid usage of `ensurePermissions` inside of nixpkgs. --- .../modules/services/databases/postgresql.nix | 46 ++++++++++++++++++- 1 file changed, 45 insertions(+), 1 deletion(-) diff --git a/nixos/modules/services/databases/postgresql.nix b/nixos/modules/services/databases/postgresql.nix index 3d2205b63555f..1c5de85bf2bc4 100644 --- a/nixos/modules/services/databases/postgresql.nix +++ b/nixos/modules/services/databases/postgresql.nix @@ -165,6 +165,33 @@ in ''; }; + ensurePermissions = mkOption { + type = types.attrsOf types.str; + default = {}; + visible = false; # This option has been deprecated. + description = lib.mdDoc '' + This option is DEPRECATED and should not be used in nixpkgs anymore, + use `ensureDBOwnership` instead. It can also break with newer + versions of PostgreSQL (≥ 15). + + Permissions to ensure for the user, specified as an attribute set. + The attribute names specify the database and tables to grant the permissions for. + The attribute values specify the permissions to grant. You may specify one or + multiple comma-separated SQL privileges here. + + For more information on how to specify the target + and on which privileges exist, see the + [GRANT syntax](https://www.postgresql.org/docs/current/sql-grant.html). + The attributes are used as `GRANT ''${attrValue} ON ''${attrName}`. + ''; + example = literalExpression '' + { + "DATABASE \"nextcloud\"" = "ALL PRIVILEGES"; + "ALL TABLES IN SCHEMA public" = "ALL PRIVILEGES"; + } + ''; + }; + ensureDBOwnership = mkOption { type = types.bool; default = false; @@ -441,6 +468,17 @@ in } ]; + # `ensurePermissions` is now deprecated, let's avoid it. + warnings = lib.optional (any ({ ensurePermissions, ... }: ensurePermissions != {}) cfg.ensureUsers) " + `services.postgresql.*.ensurePermissions` is used in your expressions, + this option is known to be broken with newer PostgreSQL versions, + consider migrating to `services.postgresql.*.ensureDBOwnership` or + consult the release notes or manual for more migration guidelines. + + This option will be removed in NixOS 24.05 unless it sees significant + maintenance improvements. + "; + services.postgresql.settings = { hba_file = "${pkgs.writeText "pg_hba.conf" cfg.authentication}"; @@ -552,7 +590,12 @@ in ${ concatMapStrings (user: - let + let + userPermissions = concatStringsSep "\n" + (mapAttrsToList + (database: permission: ''$PSQL -tAc 'GRANT ${permission} ON ${database} TO "${user.name}"' '') + user.ensurePermissions + ); dbOwnershipStmt = optionalString user.ensureDBOwnership ''$PSQL -tAc 'ALTER DATABASE "${user.name}" OWNER TO "${user.name}";' ''; @@ -564,6 +607,7 @@ in userClauses = ''$PSQL -tAc 'ALTER ROLE "${user.name}" ${concatStringsSep " " clauseSqlStatements}' ''; in '' $PSQL -tAc "SELECT 1 FROM pg_roles WHERE rolname='${user.name}'" | grep -q 1 || $PSQL -tAc 'CREATE USER "${user.name}"' + ${userPermissions} ${userClauses} ${dbOwnershipStmt} From d57926c0b62f4cdbaf005f5410a5c795e063aff3 Mon Sep 17 00:00:00 2001 From: Raito Bezarius Date: Mon, 13 Nov 2023 17:11:06 +0100 Subject: [PATCH 3/8] nixos/postgresql: improve the assertions for equality of DB user and DB name It is hard to figure out which one is offending without the database name. --- .../modules/services/databases/postgresql.nix | 23 ++++++++----------- 1 file changed, 10 insertions(+), 13 deletions(-) diff --git a/nixos/modules/services/databases/postgresql.nix b/nixos/modules/services/databases/postgresql.nix index 1c5de85bf2bc4..a9067d5974a95 100644 --- a/nixos/modules/services/databases/postgresql.nix +++ b/nixos/modules/services/databases/postgresql.nix @@ -455,19 +455,16 @@ in config = mkIf cfg.enable { - assertions = [ - { - assertion = all - ({ name, ensureDBOwnership, ... }: ensureDBOwnership -> builtins.elem name cfg.ensureDatabases) - cfg.ensureUsers; - message = '' - For each database user defined with `services.postgresql.ensureUsers` and - `ensureDBOwnership = true;`, a database with the same name must be defined - in `services.postgresql.ensureDatabases`. - ''; - } - ]; - + assertions = map ({ name, ensureDBOwnership, ... }: { + assertion = ensureDBOwnership -> builtins.elem name cfg.ensureDatabases; + message = '' + For each database user defined with `services.postgresql.ensureUsers` and + `ensureDBOwnership = true;`, a database with the same name must be defined + in `services.postgresql.ensureDatabases`. + + Offender: ${name} has not been found among databases. + ''; + }) cfg.ensureUsers; # `ensurePermissions` is now deprecated, let's avoid it. warnings = lib.optional (any ({ ensurePermissions, ... }: ensurePermissions != {}) cfg.ensureUsers) " `services.postgresql.*.ensurePermissions` is used in your expressions, From 73198870cd8afe6f763b1d2e801b143d3015363e Mon Sep 17 00:00:00 2001 From: Raito Bezarius Date: Mon, 13 Nov 2023 17:14:26 +0100 Subject: [PATCH 4/8] nixos/tests/pgbouncer: do not use `ensureDBOwnership` MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit pgbouncer test is special in the sense where it actually tries to connect via SCRAM SHA, let's avoid `ensureDBOwnership` here otherwise for some reason pgbouncer will try to look in pg_shadow for the authuser… --- nixos/tests/pgbouncer.nix | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/nixos/tests/pgbouncer.nix b/nixos/tests/pgbouncer.nix index 814ca0d58865e..bb5afd35ee28f 100644 --- a/nixos/tests/pgbouncer.nix +++ b/nixos/tests/pgbouncer.nix @@ -17,18 +17,18 @@ in systemd.services.postgresql = { postStart = '' - ${pkgs.postgresql}/bin/psql -U postgres -c "ALTER ROLE testuser WITH LOGIN PASSWORD 'testpass'"; + ${pkgs.postgresql}/bin/psql -U postgres -c "ALTER ROLE testuser WITH LOGIN PASSWORD 'testpass'"; + ${pkgs.postgresql}/bin/psql -U postgres -c "ALTER DATABASE testdb OWNER TO testuser;"; ''; }; services = { postgresql = { enable = true; - ensureDatabases = [ "test" ]; + ensureDatabases = [ "testdb" ]; ensureUsers = [ { - name = "test"; - ensureDBOwnership = true; + name = "testuser"; }]; authentication = '' local testdb testuser scram-sha-256 @@ -38,7 +38,7 @@ in pgbouncer = { enable = true; listenAddress = "localhost"; - databases = { test = "host=/run/postgresql/ port=5432 auth_user=testuser dbname=test"; }; + databases = { test = "host=/run/postgresql/ port=5432 auth_user=testuser dbname=testdb"; }; authType = "scram-sha-256"; authFile = testAuthFile; }; From 7cd63bff21f67d13810f547fb02165131f9942b1 Mon Sep 17 00:00:00 2001 From: Raito Bezarius Date: Fri, 17 Nov 2023 15:03:19 +0100 Subject: [PATCH 5/8] nixos/sourcehut: do not use `ensureDBOwnership` Given that SourceHut uses unfortunate defaults for database name, it will not be realistic to fix this in time for 23.11. We will leave the workaround and leave it to SourceHut maintainers to pick up the work to clean this up after 23.11. --- .../services/misc/sourcehut/service.nix | 19 +++++++------------ 1 file changed, 7 insertions(+), 12 deletions(-) diff --git a/nixos/modules/services/misc/sourcehut/service.nix b/nixos/modules/services/misc/sourcehut/service.nix index d2cd599d3fec1..f08d5eb468719 100644 --- a/nixos/modules/services/misc/sourcehut/service.nix +++ b/nixos/modules/services/misc/sourcehut/service.nix @@ -242,15 +242,6 @@ in } cfg.nginx.virtualHost ]; }; - assertions = [ - { - assertion = srvCfg.user == srvCfg.postgresql.database; - message = '' - When creating a database via NixOS, the db user and db name must be equal! - ''; - } - ]; - services.postgresql = mkIf cfg.postgresql.enable { authentication = '' local ${srvCfg.postgresql.database} ${srvCfg.user} trust @@ -258,10 +249,13 @@ in ensureDatabases = [ srvCfg.postgresql.database ]; ensureUsers = map (name: { inherit name; - ensureDBOwnership = true; + # We don't use it because we have a special default database name with dots. + # TODO(for maintainers of sourcehut): migrate away from custom preStart script. + ensureDBOwnership = false; }) [srvCfg.user]; }; + services.sourcehut.settings = mkMerge [ { "${srv}.sr.ht".origin = mkDefault "https://${srv}.${cfg.settings."sr.ht".global-domain}"; @@ -387,10 +381,11 @@ in extraService ])) extraServices) - # Work around 'pq: permission denied for schema public' with postgres v15, until a - # solution for `services.postgresql.ensureUsers` is found. + # Work around 'pq: permission denied for schema public' with postgres v15. # See https://github.com/NixOS/nixpkgs/issues/216989 # Workaround taken from nixos/forgejo: https://github.com/NixOS/nixpkgs/pull/262741 + # TODO(to maintainers of sourcehut): please migrate away from this workaround + # by migrating away from database name defaults with dots. (lib.mkIf ( cfg.postgresql.enable && lib.strings.versionAtLeast config.services.postgresql.package.version "15.0" From f653734c4dabbc041e7fcc72149f93567b10137c Mon Sep 17 00:00:00 2001 From: Raito Bezarius Date: Fri, 17 Nov 2023 15:19:14 +0100 Subject: [PATCH 6/8] nixos/mobilizon: do not use `ensureDBOwnership` Mobilizon can have a custom database username and it is not trivial to sort out how to remove this. In the meantime, for the upcoming 23.11 release, I apply the classical workaround and defer to Mobilizon's maintainers. --- nixos/modules/services/web-apps/mobilizon.nix | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/nixos/modules/services/web-apps/mobilizon.nix b/nixos/modules/services/web-apps/mobilizon.nix index 601c2830e0e27..bb4319b51a2fa 100644 --- a/nixos/modules/services/web-apps/mobilizon.nix +++ b/nixos/modules/services/web-apps/mobilizon.nix @@ -212,12 +212,6 @@ in assertion = cfg.nginx.enable -> (cfg.settings.":mobilizon"."Mobilizon.Web.Endpoint".http.ip == settingsFormat.lib.mkTuple [ 0 0 0 0 0 0 0 1 ]); message = "Setting the IP mobilizon listens on is only possible when the nginx config is not used, as it is hardcoded there."; } - { - assertion = isLocalPostgres -> repoSettings.database == repoSettings.username; - message = '' - When creating a database via NixOS, the db user and db name must be equal! - ''; - } ]; services.mobilizon.settings = { @@ -353,12 +347,18 @@ in # Taken from here: # https://framagit.org/framasoft/mobilizon/-/blob/1.1.0/priv/templates/setup_db.eex + # TODO(to maintainers of mobilizon): the owner database alteration is necessary + # as PostgreSQL 15 changed their behaviors w.r.t. to privileges. + # See https://github.com/NixOS/nixpkgs/issues/216989 to get rid + # of that workaround. script = '' psql "${repoSettings.database}" -c "\ CREATE EXTENSION IF NOT EXISTS postgis; \ CREATE EXTENSION IF NOT EXISTS pg_trgm; \ CREATE EXTENSION IF NOT EXISTS unaccent;" + psql -tAc 'ALTER DATABASE "${repoSettings.database}" OWNER TO "${dbUser}";' + ''; serviceConfig = { @@ -378,7 +378,10 @@ in ensureUsers = [ { name = dbUser; - ensureDBOwnership = true; + # Given that `dbUser` is potentially arbitrarily custom, we will perform + # manual fixups in mobilizon-postgres. + # TODO(to maintainers of mobilizon): Feel free to simplify your setup by using `ensureDBOwnership`. + ensureDBOwnership = false; } ]; extraPlugins = with postgresql.pkgs; [ postgis ]; From 10baca4935656089dabca0f07e3ef89f1f954375 Mon Sep 17 00:00:00 2001 From: Raito Bezarius Date: Fri, 17 Nov 2023 15:52:29 +0100 Subject: [PATCH 7/8] nixos/invidious: do not use `ensureDBOwnership` Invidious uses a strange setup where the database name is different from the system username for non-explicit reasons. Because of that, it makes it hard to migrate it to use `ensureDBOwnership`, we leave it to Invidious' maintainers to pick up the pieces. --- nixos/modules/services/web-apps/invidious.nix | 9 +++++++++ nixos/tests/invidious.nix | 3 +-- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/nixos/modules/services/web-apps/invidious.nix b/nixos/modules/services/web-apps/invidious.nix index fc9c1ec06f654..e4fbc6fd93689 100644 --- a/nixos/modules/services/web-apps/invidious.nix +++ b/nixos/modules/services/web-apps/invidious.nix @@ -109,8 +109,16 @@ let # Default to using the local database if we create it services.invidious.database.host = lib.mkDefault null; + + # TODO(raitobezarius to maintainers of invidious): I strongly advise to clean up the kemal specific + # thing for 24.05 and use `ensureDBOwnership`. + # See https://github.com/NixOS/nixpkgs/issues/216989 + systemd.services.postgresql.postStart = lib.mkAfter '' + $PSQL -tAc 'ALTER DATABASE "${cfg.settings.db.dbname}" OWNER TO "${cfg.settings.db.user}";' + ''; services.postgresql = { enable = true; + ensureUsers = lib.singleton { name = cfg.settings.db.user; ensureDBOwnership = false; }; ensureDatabases = lib.singleton cfg.settings.db.dbname; # This is only needed because the unix user invidious isn't the same as # the database user. This tells postgres to map one to the other. @@ -130,6 +138,7 @@ let documentation = [ "https://docs.invidious.io/Database-Information-and-Maintenance.md" ]; startAt = lib.mkDefault "weekly"; path = [ config.services.postgresql.package ]; + after = [ "postgresql.service" ]; script = '' psql ${cfg.settings.db.dbname} ${cfg.settings.db.user} -c "DELETE FROM nonces * WHERE expire < current_timestamp" psql ${cfg.settings.db.dbname} ${cfg.settings.db.user} -c "TRUNCATE TABLE videos" diff --git a/nixos/tests/invidious.nix b/nixos/tests/invidious.nix index 582d1550fff1a..701e8e5e7a3fc 100644 --- a/nixos/tests/invidious.nix +++ b/nixos/tests/invidious.nix @@ -44,8 +44,7 @@ import ./make-test-python.nix ({ pkgs, ... }: { enable = true; initialScript = pkgs.writeText "init-postgres-with-password" '' CREATE USER kemal WITH PASSWORD 'correct horse battery staple'; - CREATE DATABASE invidious; - GRANT ALL PRIVILEGES ON DATABASE invidious TO kemal; + CREATE DATABASE invidious OWNER kemal; ''; }; }; From 82037ad0b89db3d392c49c1fdbc6b325da5586b1 Mon Sep 17 00:00:00 2001 From: Raito Bezarius Date: Fri, 17 Nov 2023 15:57:19 +0100 Subject: [PATCH 8/8] rl-2311: inform about `services.postgresql.ensurePermissions` deprecation --- nixos/doc/manual/release-notes/rl-2311.section.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/nixos/doc/manual/release-notes/rl-2311.section.md b/nixos/doc/manual/release-notes/rl-2311.section.md index d838d33e35dc0..722bcb6b90886 100644 --- a/nixos/doc/manual/release-notes/rl-2311.section.md +++ b/nixos/doc/manual/release-notes/rl-2311.section.md @@ -141,6 +141,9 @@ ## Backward Incompatibilities {#sec-release-23.11-incompatibilities} +- `services.postgresql.ensurePermissions` has been deprecated in favor of `services.postgresql.ensureUsers.*.ensureDBOwnership` which simplifies the setup of database owned by a certain system user + in local database contexts (which make use of peer authentication via UNIX sockets), migration guidelines were provided in the NixOS manual, please refer to them if you are affected by a PostgreSQL 15 changing the way `GRANT ALL PRIVILEGES` is working. `services.postgresql.ensurePermissions` will be removed in 24.05. All NixOS modules were migrated using one of the strategy, e.g. `ensureDBOwnership` or `postStart`. More about this situation can be learnt in https://github.com/NixOS/nixpkgs/pull/266270. + - `network-online.target` has been fixed to no longer time out for systems with `networking.useDHCP = true` and `networking.useNetworkd = true`. Workarounds for this can be removed.