Skip to content

abuild: add default_pyc helper

Dominique Martinet requested to merge martinetd/abuild:pyc into master 
Python by default pre-compiles cache files in __pycache__ directories,
which we currently happily install along in python packages.

Theses .pyc files are rather big and the time/space tradeoff could be
left to users if we just split these out to a -pyc subpackage.

With this default_pyc helper, one can add $pkgname-pyc to their
package's subpackages and it will automatically split off the pyc files
in a package that will be automatically installed if the virtual 'pyc'
package is installed.
Note that this does not work so easily if there already were python
subpackages, the function could be adjusted to strip off the last dash
if required but that seems rather rare.

Random data, sizes:
 - python3: currently 47MiB, split into 23M (main package) / 24M (pyc)
 - py3-markdown: currently 700KiB, 368K (main) / 288K (pyc)

Random benchmark, with python3-pyc:
    "python3 -c 'import time; print(time.strftime(\"%T\"))'"
  Time (mean ± σ):      24.5 ms ±   2.5 ms    [User: 18.4 ms, System: 6.0 ms]
  Range (min … max):    19.4 ms …  28.9 ms    148 runs

without python3-pyc (same as user without root permissions, root would
generate files on first root, for reference this command generates 184KB
of pyc files):
    -p 'rm -rf /usr/lib/python3.10/__pycache__ /usr/lib/python3.10/encodings/__pycache__' \
    "python3 -c 'import time; print(time.strftime(\"%T\"))'"
  Time (mean ± σ):      53.7 ms ±   4.3 ms    [User: 39.3 ms, System: 14.3 ms]
  Range (min … max):    47.0 ms …  65.6 ms    100 runs

Link: https://gitlab.alpinelinux.org/alpine/aports/-/issues/11906
Suggested-by: Alex Xu (Hello71) <alex_y_xu@yahoo.ca>

If this gets merged I'll be happy to add a pyc virtual package in aports and add the subpackage to at least python3 itself, but need to start somewhere.

Merge request reports