Why there is no review articles about dwarfs on the web, if it looks like better SquashFS?

[vitaly-zdanevich]

For me it looks strange. SquashFS somehow better?

Do you know some notable projects that use DwarFS? I know only Conty.

SquashFS is more popular because it integrated into Linux Kernel? Would you like to integrate DwarFS?

[mhx]

For me it looks strange. SquashFS somehow better?

SquashFS is almost 20 years older than DwarFS, so it is much more established. Better? I guess that really depends on the use case.

Do you know some notable projects that use DwarFS? I know only Conty.

There's (at least) also Tebako.

Then again, my primary use case for DwarFS is by itself, not through any other projects.

SquashFS is more popular because it integrated into Linux Kernel?

Possibly. But then again, even if it was in the Linux kernel, what about macOS and Windows?

Would you like to integrate DwarFS?

I personally would not like to integrate it. That's not because I wouldn't like it to be part of the kernel, but because I would rather spend my time improving DwarFS.

But about your original question...

Why there is no review articles about dwarfs on the web, if it looks like better SquashFS?

I have no idea! :)

It's discussed on HN about once every two years (which is by far what has been driving most people to this project) and the occasional article on other sites with rather insignificant reach.

I guess I just suck at marketing.

[vitaly-zdanevich]

I just suck at marketing.

Best marketing - is a good product.

[vitaly-zdanevich]

I added link to your repo at https://en.wikipedia.org/wiki/SquashFS#See_also

[mhx]

I guess here are some of the actual issues that might prevent DwarFS from being more popular:

• It's not (yet) available as a Debian/Ubuntu package.
• Using it on Windows is not for the faint of heart unless you're coming from a Unix / command-line background.
• Probably the dominant use case for SquashFS is live images. I honestly haven't seen SquashFS being used for anything else. That use case is significantly simplified by SquashFS being in the kernel for 20+ years. It seems that it should be possible to use FUSE-based file systems for live images (at least there's an option to disable FUSE in the live-boot docs). Then again, I'm not sure that DwarFS would be significantly better at that particular use-case. It would certainly compress better, but likely not by more than 20-30% (that's the numbers I got from a raspbian image). It might be worth trying to get it to work though! I can think of certain cases that would definitely benefit from it, e.g. a live image with hundreds of different version of a certain toolchain.

There are a few things that I could imaging could help the popularity of DwarFS. For example, using it as a storage driver for docker seems like a really interesting idea.

[vitaly-zdanevich]

[mhx]

Thanks, can I download your repacks, to test with real USB?

So, the "repack" is really just the contents of the SquashFS file system contained in the image, so to really test this, you'd have to tweak the rest of the ISO to actually work with the new DwarFS image. I have no idea what is needed for that, though. It would at least require that the DwarFS FUSE driver is present.

Creating the "repack" is something you can easily do yourself from the ISO:

$ mkdir gentoo-live
$ mkdir gentoo-live-filesystem
$ sudo mount livegui-amd64-20250216T164837Z.iso gentoo-live
$ sudo mount gentoo-live/image.squashfs gentoo-live-filesystem
$ sudo mkdwarfs -i gentoo-live-filesystem -o image.dwarfs --categorize --with-devices --with-specials

[vitaly-zdanevich]

Creating the "repack" is something you can easily do yourself from the ISO:

Thanks, but as you said, this is not enough to boot...

[vitaly-zdanevich]

It would at least require that the DwarFS FUSE driver is present

Can you try that please? I do not know how :(
By editing of Gentoo script that selects packages to install for the ISO?

[mhx]

It would at least require that the DwarFS FUSE driver is present

Can you try that please? I do not know how :( By editing of Gentoo script that selects packages to install for the ISO?

I don't even know where to start. The driver is readily available from the release page here on GitHub, but that's likely just 2% of what's needed. I have zero experience with the whole process of building a bootable Linux image. It'd be best to collaborate with someone who has some insight into this process.

[immolo]

Has anyone packaged dwarfs for Gentoo yet? I can build an ISO with support for the FUSE driver from there and should be able to help with the debugging.

[immolo]

As I wrote before, this is literally what I did:
Sorry, I meant the conversion part to dwarfs and what the boot args need to be.

[mhx]

the conversion part to dwarfs

See here.

what the boot args need to be

I have absolutely no idea. :)

I literally just changed image.squashfs to image.dwarfs in grub.cfg, well aware that this is certainly not enough.

To mount the image.dwarfs, you would need to do something along the lines of

$ dwarfs image.dwarfs /mountpoint -oallow_other

But where, when, and who would have to run that — I'm completely clueless. Most likely in some (dracut?) script that currently does mount image.squashfs /mountpoint.

[mhx]

The dwarfs binary used to perform the mount would be this one:

$ wget https://github.com/mhx/dwarfs/releases/download/v0.10.2/dwarfs-0.10.2-Linux-x86_64-clang.tar.xz
$ tar xf dwarfs-0.10.2-Linux-x86_64-clang.tar.xz
$ cp dwarfs-0.10.2-Linux-x86_64-clang/sbin/dwarfs /somewhere-on-the-iso/

EDIT: Or simply /usr/sbin/dwarfs if you've installed sys-fs/dwarfs-bin.

[immolo]

the conversion part to dwarfs

See here.

what the boot args need to be

I have absolutely no idea. :)

I literally just changed image.squashfs to image.dwarfs in grub.cfg, well aware that this is certainly not enough.

To mount the image.dwarfs, you would need to do something along the lines of

$ dwarfs image.dwarfs /mountpoint -oallow_other

But where, when, and who would have to run that — I'm completely clueless. Most likely in some (dracut?) script that currently does mount image.squashfs /mountpoint.

Thanks, you are on the same page as me with what we need to do it seems.

I don't have the answers right now, however my work are quite keen to see if your results match when put into production and are looking at putting man hours into this. I think we should hold off while I wait for the green light for now.

I better add a disclaimer that this is my opinion only and to manage expectations, we would be looking into improving support for Gentoo with upstream patches wherever possible so other distributions can take advantage of our work for their needs also.

[mhx]

I don't have the answers right now, however my work are quite keen to see if your results match when put into production and are looking at putting man hours into this. I think we should hold off while I wait for the green light for now.

Exciting! I'm in no rush and really looking forward to seeing someone take a look at this.

I better add a disclaimer that this is my opinion only and to manage expectations, we would be looking into improving support for Gentoo with upstream patches wherever possible so other distributions can take advantage of our work for their needs also.

A proof of concept would be fantastic. I've always wondered how DwarFS would work for a live filesystem. :)