-
Notifications
You must be signed in to change notification settings - Fork 10
Some comments on documentation #436
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: TDE_REL_17_STABLE
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,7 @@ | ||
# pg_waldump | ||
|
||
AÅ: We should probably include a notice about WAL encrypting being BETA and this information is subject to change | ||
|
||
[`pg_waldump` :octicons-link-external-16:](https://www.postgresql.org/docs/current/pgwaldump.html) is a tool to display a human-readable rendering of the Write-Ahead Log (WAL) of a PostgreSQL database cluster. | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added this note: The WAL encryption feature is currently in beta and is not effective unless explicitly enabled. It is not yet production ready. Do not enable this feature in production environments. |
||
To read encrypted WAL records, `pg_waldump` supports the following additional arguments: | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -87,6 +87,7 @@ The initial decision on what file to encrypt is based on the table access method | |
The principal key is used to encrypt the internal keys. The principal key is stored in the key management store. When you query the table, the principal key is retrieved from the key store to decrypt the table. Then the internal key for that table is used to decrypt the data. | ||
|
||
### WAL encryption | ||
AÅ: We should mention this is a BETA feature and subject to change. | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added a note: WAL encryption is currently in beta and is not effective unless explicitly enabled. It is not yet production ready. Do not enable this feature in production environments. |
||
WAL encryption is done globally for the entire database cluster. All modifications to any database within a PostgreSQL cluster are written to the same WAL to maintain data consistency and integrity and ensure that PostgreSQL cluster can be restored to a consistent state. Therefore, WAL is encrypted globally. | ||
|
||
|
@@ -117,6 +118,7 @@ The support of other encryption mechanisms such as AES256 is planned for future | |
## Is post-quantum encryption supported? | ||
|
||
No, it's not yet supported. In our implementation we reply on OpenSSL libraries that don't yet support post-quantum encryption. | ||
AÅ: Afaik newer versions of openssl support post-quantum algorithms. Maybe we should use some better excuse or mention what version we use? | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Updated the paragraph to: No, pg_tde does not support post-quantum encryption. |
||
## Can I encrypt an existing table? | ||
|
||
|
@@ -134,6 +136,7 @@ You must restart the database in the following cases to apply the changes: | |
|
||
* after you enabled the `pg_tde` extension | ||
* to turn on / off the WAL encryption | ||
AÅ: WAL encrytion is not yet production ready and should probably not be casually mentioned as something you can "turn on". | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. updated this to turn on/off line to:
|
||
After that, no database restart is required. When you create or alter the table using the `tde_heap` access method, the files are marked as those that require encryption. The encryption happens at the storage manager level, before a transaction is written to disk. Read more about [how tde_heap works](index/table-access-method.md#how-tde_heap-works). | ||
|
||
|
@@ -150,6 +153,7 @@ In `pg_tde`, multi-tenancy is supported via a separate principal key per databas | |
To control user access to the databases, you can use role-based access control (RBAC). | ||
|
||
WAL files are encrypted globally across the entire PostgreSQL cluster using the same encryption keys. Users don't interact with WAL files as these are used by the database management system to ensure data integrity and durability. | ||
AÅ: Again, WAL encryption hould probably not be mentioned as if it was a feature already. | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. commented the above paragraph out for now regarding WAL |
||
## Are my backups safe? Can I restore from them? | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -10,15 +10,18 @@ The following features are available for the extension: | |
* Index data for encrypted tables | ||
* TOAST tables | ||
* Temporary tables created during database operations | ||
AÅ: Created how? | ||
|
||
!!! note | ||
Metadata of those tables is not encrypted. | ||
|
||
* Global Write-Ahead Log (WAL) encryption for data in both encrypted and non-encrypted tables | ||
AÅ: WAL encryption should not be mentioned as a feature since we don't yet have it in more than a beta preview. | ||
* Single-tenancy support via a global keyring provider | ||
* Multi-tenancy support | ||
* Table-level granularity for encryption and access control | ||
* Multiple Key management options | ||
* Logical replication of encrypted tables | ||
AÅ: We do not do anything for logical replication, do we? | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
[Overview](index/index.md){.md-button} [Get Started](install.md){.md-button} |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -10,6 +10,7 @@ However, database owners can run the “view keys” and “set principal key” | |
|
||
* `GRANT EXECUTE` | ||
* `REVOKE EXECUTE` | ||
AÅ: This should probably say `GRANT EXECUTE ON FUNCTION` and similar for revoke | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed:
|
||
## Key provider management | ||
|
||
|
@@ -52,6 +53,8 @@ The `change` functions require the same parameters as the `add` functions. They | |
|
||
Provider specific parameters differ for each implementation. Refer to the respective subsection for details. | ||
|
||
AÅ: We should probably have a notice here about the modified provider settings will need to be able to provide the exact same principal keys as the original settings. | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added the following note for this: !!! note |
||
**Some provider specific parameters contain sensitive information, such as passwords. Never specify these directly, use the remote configuration option instead.** | ||
|
||
#### Adding or modifying Vault providers | ||
|
@@ -231,6 +234,7 @@ These functions list the details of all key providers for the current database o | |
## Principal key management | ||
|
||
Use these functions to create a new principal key at a given keyprover, and to use those keys for a specific scope such as a current database, a global or default scope. You can also use them to start using a different existing key for a specific scope. | ||
AÅ: "keyprover"? | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. fixed, key provider. |
||
Princial keys are stored on key providers by the name specified in this function - for example, when using the Vault provider, after creating a key named "foo", a key named "foo" will be visible on the Vault server at the specified mount point. | ||
|
||
|
@@ -281,6 +285,7 @@ SELECT pg_tde_set_key_using_global_key_provider( | |
### pg_tde_set_server_key_using_global_key_provider | ||
|
||
Sets or rotates the server principal key using the specified global key provider. Use this function to set a principal key for WAL encryption. | ||
AÅ: Note that WAL encryption is not yet production ready | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. added the standard WAL warning here |
||
```sql | ||
SELECT pg_tde_set_server_key_using_global_key_provider( | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -36,6 +36,8 @@ The key material (actual cryptographic key) is auto-generated by `pg_tde` and st | |
!!! note | ||
This process sets the **default principal key** for the server. Any database without its own key configuration will use this key. | ||
|
||
AÅ: This is confusing to me, but maybe makes sense in the rendered version. The Example below does _not_ set a default key, does it? | ||
|
||
## Example | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I have reworded the note and the below example to properly describe the correct param:
|
||
This example is for testing purposes only. Replace the key name and provider name with your values: | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -82,6 +82,7 @@ You must do these steps for every database where you have created the extension. | |
=== "With HashiCorp Vault" | ||
|
||
The Vault server setup is out of scope of this document. | ||
AÅ: Why doesn't KMIP have a similar comment? Is KMIP more in scope than vault? Why isn't OpenBao mentioned here? | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good point! Added the same paragraph for KMIP |
||
```sql | ||
SELECT pg_tde_add_database_key_provider_vault_v2('provider-name', 'url', 'mount', 'secret_token_path', 'ca_path'); | ||
|
@@ -144,6 +145,7 @@ You must do these steps for every database where you have created the extension. | |
|
||
* `name-of-the-key` is the name of the principal key. You will use this name to identify the key. | ||
* `provider-name` is the name of the key provider you added before. The principal key will be associated with this provider. | ||
AÅ: It will not only be "associated" with it, it is where the key will be stored and fetched from. | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Updated:
|
||
<i warning>:material-information: Warning:</i> This example is for testing purposes only: | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -4,6 +4,8 @@ If you no longer wish to use TDE in your deployment, you can remove the `pg_tde` | |
|
||
Here's how to do it: | ||
|
||
AÅ: We don't think that anyone would like to drop the extension, but keep their data? Ie. should this document mention decryption of tables before dropping rather than only how to drop the tables? | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is great feedback! To ensure we covered all possible scenarios I added a new step 1:
Before removing the extension, you must either decrypt or drop all encrypted tables.
Also, added a note of warning in the intro and reworded that a bit too: !!! warning Step 2 is also updated:
|
||
1. Drop the extension using the `DROP EXTENSION` command: | ||
|
||
```sql | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -9,6 +9,7 @@ The extension is tightly integrated with Percona Server for PostgreSQL to delive | |
By using our PostgreSQL distribution, you get: | ||
|
||
- **Full encryption support** through the `tde_heap` access method, including tables, indexes, WAL data, and logical replication. | ||
AÅ: Logical replication? | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. removed |
||
- **Enhanced performance and enterprise-ready features** not available in community builds. | ||
- **Regular updates and security patches** backed by Percona’s expert support team. | ||
- **Professional support** and guidance for secure PostgreSQL deployments. | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -53,6 +53,7 @@ Here is how you can set the new default table access method: | |
1. Add the access method to the `default_table_access_method` parameter: | ||
|
||
=== "via the SQL statement" | ||
AÅ: I think "the" shouldn't be here, should it? It's also weird that it's called a "statement" here but "command" below. Maybe "via ALTER SYSTEM" is a better heading? | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Updated as such:
|
||
Use the `ALTER SYSTEM` command. This requires superuser or ALTER SYSTEM privileges. | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,7 @@ | ||
# Limitations of pg_tde | ||
|
||
AÅ: We should mention that WAL encryption isn't yet ready for use rather than mentioning pg_rewind specifically. | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Reorganized limitations, removed rewind mention and added WAL note as text, made small changes to RC version. Added note for KMS, improved system tables text. |
||
* Keys in the local keyfile are stored unencrypted. For better security we recommend using the Key management storage. | ||
* System tables are currently not encrypted. This means that statistics data and database metadata are currently not encrypted. | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -12,6 +12,7 @@ The `pg_tde` extension requires additional shared memory. You need to configure | |
You can configure the `shared_preload_libraries` parameter in two ways: | ||
|
||
* Add the following line to the `shared_preload_libraries` file: | ||
AÅ: Which file? | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed this: * Add the following line to the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Also updated the last note: !!! note |
||
```bash | ||
shared_preload_libraries = 'pg_tde' | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -30,6 +30,7 @@ After enabling the `pg_tde` extension for a database, you can begin encrypting d | |
``` | ||
|
||
The function returns `t` if the table is encrypted and `f` - if not. | ||
AÅ: The function returns `true` or `false`. the psql client specifically renders them as `t` or `f` however. | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Updated with: The function returns |
||
3. (Optional) Rotate the principal key. | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -3,6 +3,7 @@ | |
The `pg_tde` extension provides GUC variables to configure the behaviour of the extension: | ||
|
||
## pg_tde.wal_encrypt | ||
AÅ: Note that this shouldn't be enabled for production systems | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. added warning note |
||
**Type** - boolean <br> | ||
**Default** - off | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -9,6 +9,7 @@ Make sure you check the [list of supported platforms](install.md#__tabbed_1_1) b | |
The `pg_tde` uses memory locks (mlocks) to keep internal encryption keys in RAM, both for WAL and for user data. | ||
|
||
A memory lock (`mlock`) is a system call to lock a specified memory range in RAM for a process. The maximum amount of memory that can be locked differs between systems. You can check the current setting with this command: | ||
AÅ: This isn't necessarily true. Maybe OpenSSL does this under the hood? We don't do it in our code. | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hmm, we have two options here, just mention that you can check the system limit using the command if user uses mlock, or this: A memory lock ( You can check the current system limits for locked memory using: ulimit -l Thoughts? |
||
```bash | ||
ulimit -a | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So, I wanted to mention that architecture is similar to design here, so I thought why not combine these two. Added the following:
This topic outlines the long-term architectural goals and intended design for
pg_tde
. While many of the described components are implemented, however some features are still under active development and may not yet be available in the current release.Let's break down the key building blocks of this design.