[3.17] Rework device manager handling, split busybox-initscripts

It all started in the 3.16 merge window with a merge request that sparked a lot more pushback than I was expecting.

Granted, the timing was bad, and the proposed changes had more impact than I realized. So @ncopa cherry-picked a few non-controversial changes that could be made before 3.16, and the remainder of the MR was shelved.

Today I'm coming back to discuss changes in the rest of this MR (which I just reworked), that I think would be beneficial for Alpine.

• Split mdev-conf and mdev-openrc from busybox-initscripts.
• Make mdev-openrc, mdevd-openrc and udev-init-scripts-openrc all provide the dev-initscripts virtual package.
• Make alpine-base depend on dev-initscripts.

The goal is to have mdev, mdevd and udev all symmetrical, instead of having mdev be privileged because it's provided by busybox and the other device managers be hacks around it. (Eventually, in the future, I would like mdevd to become the default because it can be paired with libudev-zero to provide full udev emulation - whereas mdev cannot - but this is not the point here.)

As @ncopa pointed out, the changes require additional modifications to mkinitfs, which I am willing and able to provide manpower for: implementing the fixes, testing that things don't break, and communicating clearly and efficiently with other testers. I think the reduction in ad-hocness, which means a reduction in technical debt, is worth the effort in the long run - although @ncopa was 100% right to refuse during the 3.16 merge window.

More generally, and this is more a matter of opinion and totally debatable, I would like functionality to be progressively stripped from busybox-initscripts, which is a package that gathers a bunch of miscellaneous policy scripts that are only related by the fact that their mechanism is provided by busybox. I don't think this package makes sense from a semantics point of view; it is more logical to provide the policy scripts classified by service, no matter whether or not the implementation of the service is done by busybox. To me, ideally, busybox-initscripts would be empty, and we'd have virtual packages for every service that is currently defined in it, so support for alternative implementations can be added over time. This would also ease the path to getting out of busybox, or at least providing alternative coreutils/low-level utilities implementations, is there is ever a will from Alpine to do so.

Is there anything I can do to make sure discussions are happening? I don't want the issue to stay in limbo until the 3.17 merge window and then omg it will be too late let's postpone it to 3.18, rinse, repeat...

The goal is to have mdev, mdevd and udev all symmetrical, instead of having mdev be privileged because it's provided by busybox and the other device managers be hacks around it.

How is mdev privileged? Because it is the default?

The idea has always been to use a simple and minimal default. busybox provides that. The allow users to replace the parts with other implementations when the minimal default is not sufficient. This has worked fairly well and is simple, even if some of it is hackish (the busybox symlink creation for example).

As it is now, you can replace busybox mdev with either mdevd or udev. Not sure what the value of having them all symmetrical is?

mdev isn't privileged because it's the default, it is privileged because it's included in busybox and busybox is handled differently from other packages; it's assumed to be always there, several early openrc scripts are hardcoded to use busybox commands, etc. mdev is but one example of this.

I don't mind mdev being the default, but I do mind that switching away from the default always feels hackish, like other implementations of a service are second-class citizens. When you look at the current boot scripts, it is very obvious that they're made for mdev, and other device managers are an afterthought. Same with acpid, and really all the services that are defined in busybox-initscripts.

We can talk about busybox-initscripts in another thread (this package really feels like an ugly ad-hoc mess to me and I want it gone eventually); I'll focus on mdev here.

It is good that Alpine provides a simple and minimal default. I totally agree with that. But I don't think this default should be handled differently from other implementations. Every supported configuration should be supported equally. And I'm not only making an ideological case, but also a practical case: having all implementations symmetrical is cleaner. It reduces technical debt, as in the amount of ad-hoc knowledge that someone looking at the Alpine packaging has to learn.

I totally understand that as a maintainer, you wish to be as conservative as possible, in order to minimize the work taken on by the team. This makes sense for a lot of changes, because most changes are at best debt-neutral. This is not the case here; my goal is essentially janitorial, I want the package structure to become more logical and more maintainable. You have a blob of goo that works; putting an axe to that blob is scary, I get it, but if the result is a couple of nice and square goo cubes, they should stack more easily.

That was for the philosophy, now let's talk about mdev in particular. I don't particularly like to toot my own horn, but I feel like perception and reality are in significantly different places here.

mdev sucks.

mdev was hacked by landley at a time when hotplug helpers were all we had. It was entirely designed around being a hotplug helper, and its code is proof of it. Then it grew organically depending on people's needs; features were added here and there; and even now that there's a -d option to run it as a daemon, it retains the hotplug-based design. (Funny trivia bit about mdev -d: it potentially leaks memory on every event, because it modifies the main environment on every event, and you can't do that without risking leaks.)

On every event, mdev is spawned as a process (if run as a hotplug helper), parses its config file and runs through a complex and spaghetti decision tree to handle the event - I know, because I studied it in detail in order to replicate its exact functionality in mdevd and it was anything but roses and butterflies. And when several events arrive at the same time, either they can be handled concurrently, which sometimes does the wrong thing (e.g. when you get an add and a change event, you need to handle add before change), or the admin needs to have told mdev to serialize, which it does by busylooping and failing after two seconds if the event queue hasn't cleared up by then. Yeah, seriously, it is broken, and we're lucky that nobody notices.

Oh, wait. You did notice that the coldplug was too slow on some devices, and you had to write specific code in nlplug-findfs to do what you wanted because mdev was unusable for your use case.

Speaking of coldplug, mdev -s doesn't even perform a complete coldplug and needs an additional shell snippet to find all devices. This has been reported to the busybox ML a couple times. Has it been fixed? Crickets. Nobody cares. Yay maintainership? (Denys does what he can, but the amount of tech debt that has accumulated in busybox over the years is just too big for one person to handle.)

So when you say that mdev is a simple and minimal default, I disagree. It is minimal all right, but it's not simple and it is not even correct. Know what a simple, minimal and correct default is? mdevd.

I wrote mdevd specifically to address everything that mdev did wrong. Primarily the serialization thing, but also all the other things I mentioned above. And you know the best part? mdevd is not heavier in resource use than mdev. It's just as light. Yes, it's a long-lived process, so it commits some RAM, but that RAM - and a higher amount of it - would be used anyway while handling events. Once mdevd has started and commited its RAM, it never needs to allocate more, so event handling never fails - as opposed to mdev, which needs an additional process on every event, and also mallocs every day of the week.)

Basically, now that mdevd exists, there isn't a single technical reason anymore to use mdev. It's only the default for historical reasons, because nobody has bothered to look into it yet, because few people realize how bad mdev is, and because there is a cost to using something else because the packaging isn't symmetrical. As you rightly pointed out, device management is a can of worms that nobody wanted to open - well, it's time, and I got the PPE and the deworming chemicals.

The ideal default device manager for Alpine isn't mdev (which is ideal for nobody). It's mdevd, and every possible benchmark will agree. But I don't care about the default at this point, all I care about is making the playing field fair, and ultimately the goal is not only to ensure that Alpine provides better software, it's also to reduce tech debt and make sure that Alpine is not so tied to sucky software anymore.

added agenda:scheduled label

The TSC has discussed this issue at today's meeting and has concluded that there is a general need to begin decoupling hardcoded preferences for BusyBox from the distribution.

Accordingly, the busybox-initscripts package will be dropped, with remaining openrc scripts moved to busybox-openrc, busybox-mdev-openrc and busybox-extras-openrc. The busybox-mdev-openrc package will provide dev-openrc. Other packages implementing alternative device managers may also provide dev-openrc.

The TSC requests assistance with the development of a regression testing framework for testing mdev and alternative device managers.

The TSC will follow up on implementation status of the modified system change proposal in exactly one month.

mentioned in commit aports@57536463

added agenda:follow-up label and removed agenda:scheduled label

mentioned in merge request aports!37834 (merged)

mentioned in merge request aports!34459 (closed)

mentioned in issue #55

switching away from the default always feels hackish, like other implementations of a service are second-class citizens.

Please see #55 (comment 257899) for my thoughts on this.

The implementation of the system change proposal, aports!37834 (merged), was just merged. Thanks @psykose and @ariadne.

Over time I intend to keep splitting functionality from busybox-openrc and making it more predictable and maintainable to provide alternatives. The discussion continues in #55.

For now the mdev part is done, closing this.

closed