Implement /usr merge

Hello everybody,

I am Pablo Correa Gómez, part of the GNOME team here in alpine, but also part of postmarketOS' core team, and talking on behalf of all of us. We would like to revive the discussion initiated in the mailing list some time ago about /usr merge.

We believe there are a lot of benefits to merging /usr and we would love to see it implemented in Alpine. Some of the reasons we think it's a good idea:

• It is basically what everybody else is doing. It simplifies the system hierarchy, and removes a common source of annoying-to-debug issues, e.g: aports!44981 (merged)
• It aggregates all vendor supplied system resources in a single place. So it can be very easily mounted read-only or over the network, backed up, etc.
• Regardless of what other distros are doing, split-usr has been certainly a source of complexity for us as Alpine users and downstream maintainers. And even in the low-resource systems we maintain, we do not experience any benefit from it. We wonder others won't feel much different.

We hope these points are also appealing to you

In addition, with the advent of immutable Linux distros, verified boot features, and software like flatpak, we've been thinking a lot about the direction of postmarketOS going forward. And as a part of that, we've been investigating providing a version of postmarketOS that uses systemd as the init system. We understand that this has been a point of major contention for Alpine in the past, and we hope we can count on your discretion until we make this public.

The short of it is, systemd no longer supports running on systems with split-usr. Hence, having/usr merge in Alpine would greatly lower our maintenance threshold, and avoid lots of downstream hacks. That way, we can continue to remain as close as possible to alpine as our stack allows (something we believe has been hugely mutually beneficial in the past years).

This add an extra reason for us to implement /usr merge, and one that actually prompts us to do the work.

Right now, we have a PoC working, but with lots of downstream hacks. If this were to be considered, we offer:

• To provide the necessary patches in aports and alpine-related packages to make this work.
• To extensively test the end result and the upgrade path

So we want to know if you'd be happy with /usr merge being implemented in alpine. And if, in case we had such an implementation, there would be appetite to merge it. We understand it's hard to promise up-front, but we also want to avoid doing the work if you would be strongly opposed to it.

Best and happy hacking, Pablo and the rest of the postmarketOS team (@calebccff, @craftyguy, @dylanvanassche, @Newbyte, @PureTryOut, @ollieparanoid, @z3ntu).

Relevant work to be done to implement the /usr merge as agreed by the TSC

• Adjust abuild to warn about files in /lib, /bin, /sbin, /lib64, and /usr/sbin: abuild!308 (merged)
• Start migrating over all the obvious/basic packages (e.g. busybox, coreutils, tar, etc)
  • APK: aports!71310 (merged)
  • alpine-conf: aports!71410 (merged) and aports!71449 (merged)
  • linux-pam and deps: aports!71422 (merged)
  • os-release: aports!71504 (merged)
  • busybox: we've decided to move every utility under /usr/bin. There is already the config option INSTALL_NO_USR. ncopa had the idea to add something like INSTALL_USR_BIN that ensures all binaries to to /usr/bin. Alpine needs this (unlike other distros) because Alpine has this magic with replacing busybox programs with “proper” programs if you install e.g. coreutils. This probably means also fixing package of every package that busybox can replace!
  • Libraries (en-masse): aports!71694 (merged)
  • Libraries non-breaking: aports!71907 (merged)
  • musl, gcompat, shadow
  • mdev-conf
  • alpine-baselayout: aports!71960 (merged)
  • modules-load.d: aports!71958 (merged)
• apk-tools patches:
  • Move apk-tools data from /lib/apk to /var/lib/apk: apk-tools!203 (merged)
  • Allow apk-tools to not override symlinks: apk-tools!155 (closed)
• Changes to alpine mkinitfs/initramfs:
  • Make sure it can find kernel modules under /usr/lib
  • Make sure that it can find requested binaries under /usr/bin (and maybe also /usr/sbin?), and avoid hard-coded paths. This also needs to be patched in pmOS mkinitfs
  • Mount /usr if it exists in /etc/fstab: mkinitfs!178 (merged)
• Make alpine-base support /usr merge (and keep non-/usr merge support), make new Alpine rootfs
• Create a meta-package / trigger to handle migrating over firmware and kernels (when those packages move from /lib/firmware to /usr/lib/firmware (and same for modules) we need to keep the old paths as symlinks until we actually make /lib itself a symlink)
  • kmod trigger aports!71426 (merged)
  • other triggers: aports!71711 (closed)
• Write and package a usr-merge tool to "do" the /usr merge, with safety checks in place (e.g. we shouldn't attempt to turn /bin into a symlink if there are things in there that conflict with /usr/bin/)
• Make the usr-merge package a dependency of alpine-base