Storage: Per project image and backup storage options #14582
Conversation
Signed-off-by: Mark Bolton <[email protected]>
Signed-off-by: Mark Bolton <[email protected]>
Signed-off-by: Mark Bolton <[email protected]>
Signed-off-by: Mark Bolton <[email protected]>
Signed-off-by: Mark Bolton <[email protected]>
Signed-off-by: Mark Bolton <[email protected]>
github-actions
bot
added
Documentation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| // Create images and backups directories, and symlink to the daemon path by default. | ||
| imageDaemonPath := shared.VarPath("images", "daemon", fmt.Sprintf("project_%s", project.Name)) | ||
| imageSymPath := shared.VarPath("images", fmt.Sprintf("project_%s", project.Name)) | ||
| err = os.MkdirAll(imageDaemonPath, 0700) |
Check failure
Code scanning / CodeQL
Uncontrolled data used in path expression High
| if err != nil { | ||
| return response.InternalError(fmt.Errorf("Failed to create directory %q: %w", imageDaemonPath, err)) | ||
| } | ||
| err = os.Symlink(imageDaemonPath, imageSymPath) |
Check failure
Code scanning / CodeQL
Uncontrolled data used in path expression High
| if err != nil { | ||
| return response.InternalError(fmt.Errorf("Failed to create directory %q: %w", imageDaemonPath, err)) | ||
| } | ||
| err = os.Symlink(imageDaemonPath, imageSymPath) |
Check failure
Code scanning / CodeQL
Uncontrolled data used in path expression High
| backupsSymPath := shared.VarPath("backups", fmt.Sprintf("project_%s", project.Name)) | ||
| backupsCustomPath := filepath.Join(backupsSymPath, "custom") | ||
| backupsInstancesPath := filepath.Join(backupsSymPath, "instances") | ||
| err = os.MkdirAll(backupsDaemonPath, 0700) |
Check failure
Code scanning / CodeQL
Uncontrolled data used in path expression High
| if err != nil { | ||
| return response.InternalError(fmt.Errorf("Failed to create directory %q: %w", backupsDaemonPath, err)) | ||
| } | ||
| err = os.Symlink(backupsDaemonPath, backupsSymPath) |
Check failure
Code scanning / CodeQL
Uncontrolled data used in path expression High
| if err != nil { | ||
| return response.InternalError(fmt.Errorf("Failed to create directory %q: %w", backupsCustomPath, err)) | ||
| } | ||
| err = os.MkdirAll(backupsInstancesPath, 0700) |
Check failure
Code scanning / CodeQL
Uncontrolled data used in path expression High
| if err != nil { | ||
| return fmt.Errorf("Failed to create the new symlink at %q: %w", shared.VarPath(storageType), err) | ||
| // Create the new symlink. | ||
| err = os.Symlink(destPath, shared.VarPath(storageType, "daemon")) |
Check failure
Code scanning / CodeQL
Uncontrolled data used in path expression High
| } | ||
|
|
||
| // Create the new symlink. | ||
| err = os.Symlink(destPath, shared.VarPath(storageType, fmt.Sprintf("project_%s", projectName))) |
Check failure
Code scanning / CodeQL
Uncontrolled data used in path expression High
| if err != nil { | ||
| return fmt.Errorf("Failed to create the new symlink at %q: %w", shared.VarPath(storageType), err) | ||
| // Create the new symlink. | ||
| err = os.Symlink(destPath, shared.VarPath(storageType, "daemon")) |
Check failure
Code scanning / CodeQL
Uncontrolled data used in path expression High
| } | ||
|
|
||
| // Create the new symlink. | ||
| err = os.Symlink(destPath, shared.VarPath(storageType, fmt.Sprintf("project_%s", projectName))) |
Check failure
Code scanning / CodeQL
Uncontrolled data used in path expression High
| @@ -2535,3 +2535,7 @@ Adds a new {config:option}`device-unix-hotplug-device-conf:ownership.inherit` co | |||
| ## `unix_device_hotplug_subsystem_device_option` | |||
|
|
|||
| Adds a new {config:option}`device-unix-hotplug-device-conf:subsystem` configuration option for `unix-hotplug` devices. This adds support for detecting `unix-hotplug` devices by subsystem, and can be used in conjunction with {config:option}`device-unix-hotplug-device-conf:productid` and {config:option}`device-unix-hotplug-device-conf:vendorid`. | |||
|
|
|||
| ## `project_daemon_storage` | |||
| This introduces two new configuration keys `storage.images_volume` and `storage.backups_volume` to the project level to allow for a storage volume | |||
There was a problem hiding this comment.
| This introduces two new configuration keys `storage.images_volume` and `storage.backups_volume` to the project level to allow for a storage volume | |
| This introduces two new configuration keys, `storage.images_volume` and `storage.backups_volume`, to the project level. These keys allow for a storage volume |
|
|
||
| ## `project_daemon_storage` | ||
| This introduces two new configuration keys `storage.images_volume` and `storage.backups_volume` to the project level to allow for a storage volume | ||
| on an existing pool be used for storing the project-wide images and backups artifacts. |
There was a problem hiding this comment.
| on an existing pool be used for storing the project-wide images and backups artifacts. | |
| on an existing pool to be used for storing the project-wide images and backups artifacts. |
This PR adds support for per-project image and backup storage options. We maintain the option to select storage options at the server level, and add a more fine-grained option to select this at the project level.
Overview: We currently support configuration of storage.backups_volume and storage.images_volume on the server level. We would like to make this option more granular by introducing the ability to configure it on a per-project basis. We will continue to support the server-level configuration, but project-level configuration will take priority.
If these options are configured on a given project, then all backup/image tarballs created for resources within said project will be stored in the selected volume. If not, then we will default to the config set at the server level (if any).
Implementation:
We will maintain support for
storage.[images,backups]_volumeon the server level, and add this config option at the project level. Users may specifystorage.images_volumeandstorage.backups_volumeat the project level, which will ensure images or backups for the given project are stored on the specified storage volume. These config keys can be changed, and data will move accordingly.Images and backups will continue to be stored in the images and backups directory. However, we need some way to be able to store images and backups for a certain projects on a specified storage volume. We will follow the current pattern of using a symlink to the mountpoint of the specified volume. Special directories
backups/daemonandimages/daemonwill be created, and will function exactly as the images and backups directories function now -- to store images and backups that are not project specific. Ifstorage.[images,backups]_volumeis set at the server level, then these will be symlinked to the selected volume's mountpoint.For each project that is created, directories
images/project_Xandbackups/project_Xwill also be created, which will store images and backups for the given project. Ifstorage.[images,backups]_volumeis set at the project level, then the directory will be symlinked to the mountpoint of the selected storage volume. If not set on the project level, then the directory will be symlinked to[images,backups]/daemon/project_X. This follows the current implementation at the server level. When project-levelstorage.[images,backups]_volumeis changed or unset, data is moved accordingly.