booting directly into container storage instead of updating the rootfs

[RonnyPfannschmidt]

while playing around with bootc, it has always been a grievance of mine that the roo-tfs needs to be changed to actually boot into a "image"

when playing around with btrfs i took note that the Storages is basically what one would want of a root

so im wondering, what if instead of the normal root-fs, the data was used directly from the storage of container storage - like any other container - no more updating the root-fs to boot a different image

depending on the boot-loader one could go even farther (and skip extracting the target kernel/initrd, instead using a kexec when hardware can deal with reinit

the key thing for me would be that instead of updating the rootfs - the container would only extract the own kernel to the boot partition

additionally some details could be managed more graceful (imagine a usb drive with images for both x86/arm being able to boot on any system one connects)

plus passing a container storage to a vm + adding the kernel/initrd could possibly enable quick testing of images by avoiding writing the disk images instead opting to only store persistent data in the images, but using the hosts container storage to serve the system

[cgwalters]

I understand what you're thinking, but there's a few complexities here. Actually a lot. This overlaps with several things - #20 is the biggest.

the key thing for me would be that instead of updating the rootfs - the container would only extract the own kernel to the boot partition

But what specifically is the advantage of this? Are you thinking this would somehow speed things up? Use less disk space? Be easier to understand?

additionally some details could be managed more graceful (imagine a usb drive with images for both x86/arm being able to boot on any system one connects)

I am not totally sure that's possible with the way booting works with e.g. EFI regardless of containers.

plus passing a container storage to a vm + adding the kernel/initrd could possibly enable quick testing of images by avoiding writing the disk images instead opting to only store persistent data in the images, but using the hosts container storage to serve the system

A "virtiofs root" though doesn't require changing how we do things in general though. The virtiofs root path also relates to #898

[RonnyPfannschmidt]

I understand what you're thinking, but there's a few complexities here. Actually a lot. This overlaps with several things - #20 is the biggest.

the key thing for me would be that instead of updating the rootfs - the container would only extract the own kernel to the boot partition

But what specifically is the advantage of this? Are you thinking this would somehow speed things up? Use less disk space? Be easier to understand?

remove the need to imprint the content of the image to the root when booting a different image
instead of having a install to disk step necessary to boot, one would just boot a image
the primary concern is not disk space - but the ability to instantly and directly select a different boot image without needing to update the rootfs for booting

to my understanding this would in particular help for lager configurations (like universal blue derivations and other huge bootc setups)

additionally some details could be managed more graceful (imagine a usb drive with images for both x86/arm being able to boot on any system one connects)

I am not totally sure that's possible with the way booting works with e.g. EFI regardless of containers.

according to https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html#removable-media-boot-behavior the basic booting is supposed to be possible

now with current bootc - one will need one container installed to the rootfs, thus excluding the possibility of multi arch

however if a container storage was used as boot point, then one could easily put in different arch builds of the container after all they are part of the same upload manifest for any other multi platform build

plus passing a container storage to a vm + adding the kernel/initrd could possibly enable quick testing of images by avoiding writing the disk images instead opting to only store persistent data in the images, but using the hosts container storage to serve the system

A "virtiofs root" though doesn't require changing how we do things in general though. The virtiofs root path also relates to #898

the primary match of my proposal would be "boot from shared host file-system data instead of having a disks/network"
having network/ram based boot-up is a different layout, i believe it perfectly matches numerous coreos style setups with intentional small surface of the to be installed os

however i suspect for example https://universal-blue.org images are not a good match (as those can easily exceed a few gb - which isn't exactly sensible for a quickly test from ram after lan/wifi load scenario)

my current half-informed understanding is that the "easiest" match for possibly using container storage would be using a native fs driver like btrfs/zfs and then using matching subvolumes as roots instead of writing to the normal root

and/or abusing snap-shooting form a initramfs and imposing the container image subvolume as root subvolume before chrooting into the real system

an initramfs could possibly also be used to create a overlay fs mount to boot into hen using the standard driver

[cgwalters]

according to https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html#removable-media-boot-behavior the basic booting is supposed to be possible

Right, OK.

now with current bootc - one will need one container installed to the rootfs, thus excluding the possibility of multi arch

It's not "installed to the rootfs" that's really the issue here in that sense, it's the indirection we have in the classic bootloader chain from shim -> grub -> bootloader entries -> rootfs.

Anyways, now that I look I see there's an architecture key in the BLS for type 1 entries
https://uapi-group.org/specifications/specs/boot_loader_specification/#type-1-boot-loader-specification-entries

I am not sure if grub implements that. If it does, then for sure it would be possible to teach bootc to use it.

my current half-informed understanding is that the "easiest" match for possibly using container storage would be using a native fs driver like btrfs/zfs and then using matching subvolumes as roots instead of writing to the normal root

I know what you're thinking but fundamentally changing the storage stack doesn't actually help here, it's more about the early boot plumbing plus multi-installation support.

[RonnyPfannschmidt]

i did some more reading on ostree as i was unaware of its roots/root-switching concepts

im under the impression that more plumbing support would be most helpful

as long as ostree is the primary storage format, any mechanism that would allow to add new/selectable roots would help the root selection/switch-root mechanisms are pretty interesting

if unified storage gets added - using oci containers as roots wold be considered

with my expanded understanding im under the impression the key plumbing details needed are

1, a mechanism to create a ostree root from a container image (until container storage is directly supported)
2. a mechanism to extract/create a uefi entry for a kernel from such a root

this would forego adding grub in any way, my current understanding is that grub doesn't support what i have in mind

having such plumbing in place would enable all use-cases i have in mind via
a) having ostree manage the data/de-duplication (i believe this would enable very quick image/external hard-disk updates)
b) having uefi manage platform selection and details

with what i learned i believe that plumbing tools are more important than the eventual storage unification for my particular use-cases

specifically better uefi boot tooling and better ostree "imprinting?" tooling

[RonnyPfannschmidt]

additional note - would it be possible/feasible/sensible to have a storage driver for podman thats ostree backed - then adding a image to a disk would be just about the same as putting a container into a container store