Skip to content

main/mesa: consolidate subpackages

Minecrell requested to merge Minecrell/aports:mesa-megadrivers into master

Mesa uses so-called "megadrivers" to link multiple drivers into one shared library. (See e.g. blog post, presentation). This is done to allow the compiler to do more optimization at link time. Essentially there are only 4 distinct shared libraries built:

  • libmesa_dri_drivers.so ("classic" DRI drivers)
  • libgallium_dri.so (Gallium DRI drivers)
  • libgallium_drv_video.so (Gallium VAAPI drivers)
  • libvdpau_gallium.so.1.0.0 (Gallium VDPAU drivers)
  • (Note: Vulkan drivers are all built separately)

These shared libraries are then hard-linked for each driver (i965, radeon, freedreno, ...). This is visible when looking at the build log:

x86_64 (click me)
installing src/mesa-19.3.1/output/src/mesa/drivers/dri/libmesa_dri_drivers.so to pkg/mesa/usr/lib/xorg/modules/dri/i915_dri.so
installing src/mesa-19.3.1/output/src/mesa/drivers/dri/libmesa_dri_drivers.so to pkg/mesa/usr/lib/xorg/modules/dri/i965_dri.so
installing src/mesa-19.3.1/output/src/mesa/drivers/dri/libmesa_dri_drivers.so to pkg/mesa/usr/lib/xorg/modules/dri/radeon_dri.so
installing src/mesa-19.3.1/output/src/mesa/drivers/dri/libmesa_dri_drivers.so to pkg/mesa/usr/lib/xorg/modules/dri/r200_dri.so
installing src/mesa-19.3.1/output/src/mesa/drivers/dri/libmesa_dri_drivers.so to pkg/mesa/usr/lib/xorg/modules/dri/nouveau_vieux_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/radeonsi_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/nouveau_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/swrast_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/kms_swrast_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/iris_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/r300_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/r600_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/vmwgfx_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/virtio_gpu_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/vdpau/libvdpau_gallium.so.1.0.0 to pkg/mesa/usr/lib/vdpau/libvdpau_r300.so.1.0.0
installing src/mesa-19.3.1/output/src/gallium/targets/vdpau/libvdpau_gallium.so.1.0.0 to pkg/mesa/usr/lib/vdpau/libvdpau_r600.so.1.0.0
installing src/mesa-19.3.1/output/src/gallium/targets/vdpau/libvdpau_gallium.so.1.0.0 to pkg/mesa/usr/lib/vdpau/libvdpau_radeonsi.so.1.0.0
installing src/mesa-19.3.1/output/src/gallium/targets/vdpau/libvdpau_gallium.so.1.0.0 to pkg/mesa/usr/lib/vdpau/libvdpau_nouveau.so.1.0.0
aarch64 (click me)
installing src/mesa-19.3.1/output/src/mesa/drivers/dri/libmesa_dri_drivers.so to pkg/mesa/usr/lib/xorg/modules/dri/radeon_dri.so
installing src/mesa-19.3.1/output/src/mesa/drivers/dri/libmesa_dri_drivers.so to pkg/mesa/usr/lib/xorg/modules/dri/r200_dri.so
installing src/mesa-19.3.1/output/src/mesa/drivers/dri/libmesa_dri_drivers.so to pkg/mesa/usr/lib/xorg/modules/dri/nouveau_vieux_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/armada-drm_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/exynos_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/hx8357d_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/ili9225_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/ili9341_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/imx-drm_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/meson_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/mi0283qt_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/mxsfb-drm_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/pl111_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/repaper_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/rockchip_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/st7586_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/st7735r_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/stm_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/sun4i-drm_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/radeonsi_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/nouveau_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/msm_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/kgsl_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/swrast_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/kms_swrast_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/v3d_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/vc4_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/panfrost_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/etnaviv_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/tegra_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/r300_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/r600_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/virtio_gpu_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/dri/libgallium_dri.so to pkg/mesa/usr/lib/xorg/modules/dri/lima_dri.so
installing src/mesa-19.3.1/output/src/gallium/targets/vdpau/libvdpau_gallium.so.1.0.0 to pkg/mesa/usr/lib/vdpau/libvdpau_r300.so.1.0.0
installing src/mesa-19.3.1/output/src/gallium/targets/vdpau/libvdpau_gallium.so.1.0.0 to pkg/mesa/usr/lib/vdpau/libvdpau_r600.so.1.0.0
installing src/mesa-19.3.1/output/src/gallium/targets/vdpau/libvdpau_gallium.so.1.0.0 to pkg/mesa/usr/lib/vdpau/libvdpau_radeonsi.so.1.0.0
installing src/mesa-19.3.1/output/src/gallium/targets/vdpau/libvdpau_gallium.so.1.0.0 to pkg/mesa/usr/lib/vdpau/libvdpau_nouveau.so.1.0.0
installing src/mesa-19.3.1/output/src/gallium/targets/va/libgallium_drv_video.so to pkg/mesa/usr/lib/dri/r600_drv_video.so
installing src/mesa-19.3.1/output/src/gallium/targets/va/libgallium_drv_video.so to pkg/mesa/usr/lib/dri/radeonsi_drv_video.so
installing src/mesa-19.3.1/output/src/gallium/targets/va/libgallium_drv_video.so to pkg/mesa/usr/lib/dri/nouveau_drv_video.so

The APKBUILD attempts to create minimal packages by splitting the drivers into separate subpackages. Unfortunately this has exactly the opposite effect in this case:
Instead of preserving the hard links with one large Mesa package, we copy the same shared library over and over again into separate subpackages. This causes overhead for storing all the apk packages, but also when installing multiple drivers on one system.

This MR consolidates the DRI subpackages into 4 subpackages - one for each of the distinct shared libraries mentioned above. This allows the hard-links to stay working.
The end result is that we significantly reduce the disk space needed for Mesa packages, for both the apk packages and when installing multiple drivers on a system.

Here is a comparison on aarch64 (granted, barely anyone will install all drivers):

Before:

$ du -hc mesa-*.apk
...
80.0M   total

# apk add -i mesa mesa-dev mesa-dri-ati mesa-dri-etnaviv mesa-dri-freedreno mesa-dri-kmsro mesa-dri-lima mesa-dri-nouveau mesa-dri-panfrost mesa-dri-swrast mesa-dri-tegra mesa-dri-v3d mesa-dri-vc4 mesa-dri-virtio mesa-egl mesa-gbm mesa-gl mesa-glapi mesa-gles mesa-osmesa mesa-vulkan-ati mesa-xatracker
The following NEW packages will be installed:
  mesa-glapi mesa mesa-gbm mesa-egl mesa-gl mesa-gles mesa-osmesa mesa-xatracker mesa-dev mesa-dri-ati mesa-dri-etnaviv mesa-dri-freedreno mesa-dri-kmsro mesa-dri-lima mesa-dri-nouveau mesa-dri-panfrost mesa-dri-swrast mesa-dri-tegra mesa-dri-v3d mesa-dri-vc4 mesa-dri-virtio mesa-vulkan-ati
After this operation, 226 MiB of additional disk space will be used.

After:

$ du -hc mesa-*.apk
...
17.5M   total

# apk add -i mesa mesa-dev mesa-dri-classic mesa-dri-gallium mesa-egl mesa-gbm mesa-gl mesa-glapi mesa-gles mesa-osmesa mesa-va-gallium mesa-vdpau-gallium mesa-vulkan-ati mesa-xatracker
The following NEW packages will be installed:
  mesa mesa-gbm mesa-glapi mesa-egl mesa-gl mesa-gles mesa-osmesa mesa-xatracker mesa-dev mesa-dri-classic mesa-dri-gallium mesa-va-gallium mesa-vdpau-gallium mesa-vulkan-ati
After this operation, 50 MiB of additional disk space will be used.

In both cases, exactly the same files are installed. The size is reduced because we now preserve all the hard-links.

I have added some compatibility packages that should make upgrades work correctly. See !2816 (comment 61614) for an explanation.

I'm not aware of any way to prevent the Mesa build system from producing these megadrivers. I guess the only alternative option would be to build Mesa several times (with fewer drivers enabled) but this would significantly increase the build time...

Edited by Minecrell

Merge request reports