abuild issueshttps://gitlab.alpinelinux.org/alpine/abuild/-/issues2024-01-15T09:15:25Zhttps://gitlab.alpinelinux.org/alpine/abuild/-/issues/9996Alpine repo drops packages. This prevents package version pinning, and makes...2024-01-15T09:15:25ZP. CowlinatorAlpine repo drops packages. This prevents package version pinning, and makes apk non-deterministic.Alpine Linux has gained popularity as a Linux distribution that is especially good for Docker images.
What’s one of the biggest benefits of Docker? Reproducibility, including deterministic, reproducible Dockerfile builds.
You should be...Alpine Linux has gained popularity as a Linux distribution that is especially good for Docker images.
What’s one of the biggest benefits of Docker? Reproducibility, including deterministic, reproducible Dockerfile builds.
You should be able to make a Dockerfile deterministic by pinning package version numbers, so that your image is not dependent on the point in time when it was built.
Unfortunately, the Alpine package repo drops packages, even packages on "stable" branches.
Example: on 2020 March 10th, I found gcc 9.2.0-r3 on the Alpine package repository (web UI) under branch 3.11. On 2020 March 23rd, just 13 days later, my Dockerfile failed to run because the package gcc 9.2.0-r3 had been revoked from the branch 3.11 of the package repository, and was replaced with gcc 9.2.0-r4.
This makes Alpine Linux unsuitable for use in Docker images. Either your Dockerfile with pinning will "expire", or you are forced to avoid pinning package versions, which may cause unexpected behavior. When package maintainers decide to release a new version, this unexpected version will be automatically installed as soon as you rebuild your image the next time.
Compare this to PyPI or npm: No version is dropped, so version pinning works perfectly fine, no matter when you build or use your stuff.
There is a similar thread, https://gitlab.alpinelinux.org/alpine/apk-tools/issues/10661 , which Timo Teräs (@fabled) closed, based on the unconfirmed assumption that the OP was mixing an Alpine image with Alpine packages from 2 different branches.
However, in my example, both the Alpine version and the package version were on the 3.11 branch. There is no mixing.https://gitlab.alpinelinux.org/alpine/abuild/-/issues/9997Follow XDG Base Directory specifications2023-10-24T01:52:52ZBart RibbersFollow XDG Base Directory specificationsCurrently abuild write it's config file and build keys to `~/.abuild`, and puts generated packages in `~/packages`. It would be nice if it would use [the XDG Base Directory specification](https://specifications.freedesktop.org/basedir-sp...Currently abuild write it's config file and build keys to `~/.abuild`, and puts generated packages in `~/packages`. It would be nice if it would use [the XDG Base Directory specification](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html) instead so it doesn't clutter the `$HOME` directory.
The config file would go to `~/.config/abuild/abuild.conf` (if `$XDG_CONFIG_DIR` isn't set) and the keys and packages to `~/.local/share/abuild` (if `$XDG_DATA_DIRS` isn't set).https://gitlab.alpinelinux.org/alpine/abuild/-/issues/10090Move config directory to $XDG_CONFIG_HOME2023-01-12T11:08:15ZNatanael CopaMove config directory to $XDG_CONFIG_HOMEWe should move the `~/.abuild` to `~/.config/abuild`. (using `$XDG_CONFIG_HOME`)
We should also have a fallback to `~/.abuild` with a deprecation warning, but we should not auto-migrate it for the user.
If both exists, the deprecated `...We should move the `~/.abuild` to `~/.config/abuild`. (using `$XDG_CONFIG_HOME`)
We should also have a fallback to `~/.abuild` with a deprecation warning, but we should not auto-migrate it for the user.
If both exists, the deprecated `~/.abuild` should be ignored, with a warning.https://gitlab.alpinelinux.org/alpine/abuild/-/issues/10062Add default_s6 subpackage split function2023-02-11T05:38:12ZPatrycja Rosaalpine@ptrcnull.meAdd default_s6 subpackage split functionThis would allow for package maintainers to include s6 service files, either for users that would want to use s6 alongside openrc, or with regard to future migration of the whole ecosystemThis would allow for package maintainers to include s6 service files, either for users that would want to use s6 alongside openrc, or with regard to future migration of the whole ecosystemhttps://gitlab.alpinelinux.org/alpine/abuild/-/issues/10051Set GIT_CEILING_DIRECTORIES during prepare/build/check/package2022-06-27T11:43:46ZKevin DaudtSet GIT_CEILING_DIRECTORIES during prepare/build/check/packageMany build systems expect or check if they are run in a git repository, and either get information from it, like the version of the project or even modify it (installing hooks, like husky annoyingly does). As we get the source code from ...Many build systems expect or check if they are run in a git repository, and either get information from it, like the version of the project or even modify it (installing hooks, like husky annoyingly does). As we get the source code from archives, there is never an upstream repository involved, and as a result, aports is being used.
Git has an environment variable called GIT_CEILING_DIRECTORIES:
> This should be a colon-separated list of absolute paths. If set, it is a list of directories that Git should not chdir up into while looking for a repository directory
So if we set something like:
```
export GIT_CEILING_DIRECTORIES=$STARTDIR
```
then git will never see aports as a repository while building projects, preventing incorrect information leaking into projects and projects like husky damaging aports.
As abuild itself needs to get information from aports, we cannot blankly set it somewhere global. It should also be in affect when a user manually runs `abuild build` for example.https://gitlab.alpinelinux.org/alpine/abuild/-/issues/9985Allow downloading sources with Git2021-04-28T11:29:19ZBart RibbersAllow downloading sources with GitSometimes upstream projects require submodules but don't make release tarballs which include them. It would be nice to be able to clone the sources recursively in those cases instead so not every subproject has to be manually specified.
...Sometimes upstream projects require submodules but don't make release tarballs which include them. It would be nice to be able to clone the sources recursively in those cases instead so not every subproject has to be manually specified.
Besides being able to set a giturl, it should allow checking out a version or specific commit.https://gitlab.alpinelinux.org/alpine/abuild/-/issues/10060Subpackage versioning2022-08-14T17:58:30ZAntoine MartinSubpackage versioningThe following discussion from aports!28195 should be addressed:
- [ ] @jirutka started a [discussion](https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/28195#note_205465): (+14 comments)
> `pkgver` should not be redefi...The following discussion from aports!28195 should be addressed:
- [ ] @jirutka started a [discussion](https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/28195#note_205465): (+14 comments)
> `pkgver` should not be redefined for subpackages. I tried that a few years ago in some aports and it didn’t work well with the build servers (I don’t remember details). However, maybe this is a good time to open a discussion about this, because this is a quite unfortunate limitation and I’d like to overcome it.
>
> /cc @ncopa, @ariadne, @kdaudt
dotnet packages require custom pkgver within subpackages, as SDK and Runtime have different versioning schemes. This is not currently supported by buildrepo or abuild as they source only global pkgver to check if there should be a rebuild of packages, or whether old packages should be purged. I implemented a mitigation from within my aports that create a symbolic link between where the APK is expected to be by abuild and buildrepo, and where it actually is. This works for most use-cases that I could test for, except for buildrepo's purging functionality. As buildrepo purges every APK that doesn't match `(name or pkg.pkgname).."-"..pkg.pkgver.."-r"..pkg.pkgrel..".apk"`, this means it deletes any subpackage with custom pkgver.
The mitigation is as follows:
```bash
scan_symlink_targets() {
local name="$1" dir="$2"
local ver=$(pkginfo_val pkgver "$dir"/.PKGINFO)
local subpkgarch=$(pkginfo_val arch "$dir"/.PKGINFO)
[ -d "$REPODEST"/${repo:?}/${subpkgarch/noarch/$CARCH} ] || mkdir -p "$REPODEST"/$repo/${subpkgarch/noarch/$CARCH}
[ "$ver" = "$pkgver-r$pkgrel" ] || ln -sf "$REPODEST"/$repo/${subpkgarch/noarch/$CARCH}/$name-$ver.apk "$REPODEST"/$repo/${subpkgarch/noarch/$CARCH}/$name-$pkgver-r$pkgrel.apk
}
```
# Problematic functions
Through my attempts at figuring out a mitigation, I identified the following problematic functions
## abuild
* `scan_symlink_targets`: when it detects symlink dependencies between subpkgs sets a hard dependency to that specific version of subpackage. Above mitigation neutralizes scan_symlink_targets, leaving the maintainer to set those dependencies explicitely
* `apk_up2date` / `abuildindex_up2date`:root cause of subpkg issues in abuild. Both functions define expected path as `"$dir"/$subpkgname-$pkgver-r$pkgrel.apk`. Since these are defined from within a for loop for each package, `$pkgver` should rather be `$subpkgver`, but no logics are in place to do so.
* `subpkg_set`: defines above `$pkgver` variable. Logics for `$subpkgver` would be implemented here.
## lua-aports
* `db:each_need_build` / purge logics (`buildrepo`): former relies on `pkg.apk_file_exists` which checks if there's a readable file at output of `get_apk_file_path`, which relies on `get_apk_file_name`. Latter also relies on `get_apk_file_name`
* `get_apk_file_name` (`buildrepo/pkg.lua`): root function of the above functions, defines `apk_file_name` as `(name or pkg.pkgname).."-"..pkg.pkgver.."-r"..pkg.pkgrel..".apk"`. `pkg.pkgver` is presumable sourced from aport's global pkgver rather than APKINDEX. If `get_apk_file_name` sourced pkg.pkgver from subpkgver, this would likely fix the issues with buildrepo. This function is thus a root-case for issues relating to subpkg versioning.
# Implementation proposal
## aports
Similar to :arch implementation within subpackages array, this would probably be a similar strategy to define pkgver for subpackages. Thus possible variables within subpackages would be `subpkgname:subpkgsplit:subpkgver:subpkgarch` rather than `subpkgname:subpkgsplit:subpkgarch`
It would then be a matter of changing the above abuild functions, implementing most of the logics in `subpkg_set`.
## Buildrepo
I know little of lua, so while I might make an attempt at an implementation, I can't even tell how `pkg.pkgver` is sourced. The only thing I can tell is that `get_apk_file_name` *seems* to be where I need to implement the logics, but then again `db:each_name` seems to also be a candidate. This is likely something that someone more lua-savy should address.https://gitlab.alpinelinux.org/alpine/abuild/-/issues/10057RFC: implement optimization profiles2022-02-06T03:00:21ZAriadne Conillariadne@ariadne.spaceRFC: implement optimization profilesPresently, `abuild` only supports a unified set of `CFLAGS`, et al. If you want to override the `CFLAGS` used for a package, you must do it in the `APKBUILD` for that package. However, there are many packages where the default `-Os` op...Presently, `abuild` only supports a unified set of `CFLAGS`, et al. If you want to override the `CFLAGS` used for a package, you must do it in the `APKBUILD` for that package. However, there are many packages where the default `-Os` optimization setting does not make any sense: for the amount of binary size savings, we lose significant performance -- an example of this being the OpenBLAS and LAPACK libraries.
Talking about this with @jirutka we thought about the concept of an *optimization profile*: a package maintainer would specify how the build should be optimized, and `abuild` would choose the correct CFLAGS based on that profile. For example, an `APKBUILD` could have `optprofile=size` or `optprofile=performance`, which would choose different CFLAGS based on the requested profile. It would be desirable for the end-user to be able to override the default profiles in `abuild.conf`, as well.
Thinking out loud, there could be at least three profiles, but we may want additional profiles to trade hardening features for performance too:
* `size`: the current default CFLAGS: `-Os -fomit-frame-pointer`
* `balanced`: basically the same as the current default, but `-O2`
* `performance`: `-O3 -ffast-math -funroll-all-loops ...`
We probably want to tweak these profiles for specific microarchitectural optimizations, too, but that seems like something where @alxu probably knows a lot more than me.
The point of this being that we no longer are just overriding CFLAGS to get `-O2` and so on, it keeps everything more consistent with the building.https://gitlab.alpinelinux.org/alpine/abuild/-/issues/10023RFC: enable link time optimization2023-06-22T04:15:18ZJ0WIRFC: enable link time optimizationFedora, SUSE or the Firefox builds provided by Mozilla use LTO by default and the numbers look promising. LTO aims to produce smaller and faster binaries.
I suggest to try LTO as a default for Alpine.
Further reading:
- https://fedorap...Fedora, SUSE or the Firefox builds provided by Mozilla use LTO by default and the numbers look promising. LTO aims to produce smaller and faster binaries.
I suggest to try LTO as a default for Alpine.
Further reading:
- https://fedoraproject.org/wiki/LTOByDefault
- https://www.ucw.cz/~hubicka/slides/opensuse2018-e.pdf
- https://hubicka.blogspot.com/2016/03/building-libreoffice-with-gcc-6-and-lto.html#more
- https://glandium.org/blog/?p=3888
- https://wiki.ubuntu.com/ToolChain/LTOhttps://gitlab.alpinelinux.org/alpine/abuild/-/issues/9016Support GnuPG public keys as an alternative to checksums2024-03-24T16:17:48ZalgitbotSupport GnuPG public keys as an alternative to checksumsSigned packages provide more security than checksums, e.g. in the case
of corrupt mirrors or download sites.
The private key is only owned by the devs or release managers. All users
can use the well known public key to verify their dow...Signed packages provide more security than checksums, e.g. in the case
of corrupt mirrors or download sites.
The private key is only owned by the devs or release managers. All users
can use the well known public key to verify their downloads. As an
additional feature, the key can be fetched from keyservers, so
corrupt/revoked keys will throw an error.
e.g. in the case of nginx:
Fetch B0F4253373F8F6F510D42178520A9993A1C052F8 in the APKBUILD and fetch
the \*.asc together with the tarball/signed git tag.
*(from redmine: issue id 9016, created on 2018-06-16)*https://gitlab.alpinelinux.org/alpine/abuild/-/issues/10097PIE check2023-06-03T12:37:08ZGhost UserPIE checkcurrently, non-PIE binaries are not flagged/detected. perhaps they should be?
either a warning or a failure with options="!pie" would workcurrently, non-PIE binaries are not flagged/detected. perhaps they should be?
either a warning or a failure with options="!pie" would workhttps://gitlab.alpinelinux.org/alpine/abuild/-/issues/10094rootbld cannot create groups from pkggroups=2023-12-27T09:15:30ZGhost Userrootbld cannot create groups from pkggroups=with a `pkggroups=pipewire`
```
>>> pipewire: Unpacking /var/cache/distfiles/pipewire-0.3.66.tar.gz...
>>> pipewire: Creating group pipewire
abuild-addgroup: User demon is not a member of group abuild
>>> ERROR: pipewire: mkusers faile...with a `pkggroups=pipewire`
```
>>> pipewire: Unpacking /var/cache/distfiles/pipewire-0.3.66.tar.gz...
>>> pipewire: Creating group pipewire
abuild-addgroup: User demon is not a member of group abuild
>>> ERROR: pipewire: mkusers failed
>>> ERROR: pipewire: rootbld failed
```
the actual (host) user is in `abuild`https://gitlab.alpinelinux.org/alpine/abuild/-/issues/10078cleanoldpkg option is not limited to one architecture2022-12-06T16:12:19ZCarl Chavecleanoldpkg option is not limited to one architectureSome abuild options seem to be architecture constrained, like `index` for example seems to just rebuild the index for the detected arch unless you provide CHOST or CARCH:
```
server:/srv/aports/local/xlunch$ abuild index
>>> xlunch: Upd...Some abuild options seem to be architecture constrained, like `index` for example seems to just rebuild the index for the detected arch unless you provide CHOST or CARCH:
```
server:/srv/aports/local/xlunch$ abuild index
>>> xlunch: Updating the local/x86_64 repository index...
>>> xlunch: Signing the index...
server:/srv/aports/local/xlunch$ CHOST=aarch64 abuild index
>>> xlunch: Updating the local/aarch64 repository index...
>>> xlunch: Signing the index...
```
The `cleanoldpkg` option does not limit itself to a single arch and to me it's not obvious when reviewing the -h help for abuild or from the command output during execution that its cleaning all architectures, not just the detected/specified one.
```
server:/srv/aports/local/xlunch$ abuild cleanoldpkg
>>> xlunch: Cleaning all packages except 4.7.4-r1...
>>> xlunch: Updating the local/x86_64 repository index...
>>> xlunch: Signing the index...
```
The first step cleans all architectures and then the next two just update and sign the x86_64 index leaving the indexes (or indices) for the other architectures inconsistent with the content of the repo.
Maybe this is by design and this is just operator error on my part but its a foot gun that I keep shooting myself with on my personal repo. I'll experiment with some package changes on x86_64 for example and bump the pkgver but only build x86_64. Then I'll forget and run `abuild cleanoldpkg` thinking it's limited to x86_64 but it's actually deleting the package completely from other architectures because I haven't built the current pkgver for those yet. Then cron runs later and rysnc deletes all the packages out of my remote repo too.
As I said, I know the problem is mostly on my end but I don't think `cleanoldpkg` is clear about what it's doing and I don't think it's consistent with other commands like `index`.https://gitlab.alpinelinux.org/alpine/abuild/-/issues/10074dependency conflict coreutils 9.0 and ucspi-tcp6 1.0.52022-09-13T00:11:02ZTherminoel.kuntze@thermi.consultingdependency conflict coreutils 9.0 and ucspi-tcp6 1.0.5```
apk add coreutils -l
ERROR: unable to select packages:
coreutils-9.0-r2:
conflicts: ucspi-tcp6-1.05-r0[cmd:date=9.0-r2] ucspi-tcp6-1.05-r0[cmd:who=9.0-r2]
satisfies: world[coreutils]
ucspi-tcp6-1.05-r0:
conflicts: cor...```
apk add coreutils -l
ERROR: unable to select packages:
coreutils-9.0-r2:
conflicts: ucspi-tcp6-1.05-r0[cmd:date=9.0-r2] ucspi-tcp6-1.05-r0[cmd:who=9.0-r2]
satisfies: world[coreutils]
ucspi-tcp6-1.05-r0:
conflicts: coreutils-9.0-r2[cmd:date] coreutils-9.0-r2[cmd:who]
satisfies: world[ucspi-tcp6]
```
Root cause is this:
```
/ # apk --no-network info -e -a coreutils | grep who
cmd:who
cmd:whoami
usr/bin/who
usr/bin/whoami
/ # apk --no-network info -a coreutils | grep who
cmd:who
cmd:whoami
usr/bin/who
usr/bin/whoami
cmd:who=9.0-r2
cmd:whoami=9.0-r2
/ # apk --no-network info -a ucspi-tcp6 | grep who
cmd:who
usr/bin/who@
cmd:who=1.05-r0
```
The `provides` part of the package for ucspi-tcp6 contains the one for `who`, although it packages `who@`.
The problem occurs because coreutils 9.0 package "who" in a specific version now.https://gitlab.alpinelinux.org/alpine/abuild/-/issues/10073versioned cmd: providers make it impossible to use provider_priority2022-08-07T19:30:21ZGhost Userversioned cmd: providers make it impossible to use provider_prioritycurrently, the default cmd: providers have a version appended. so, for instance, both mandoc and man-db provide:
```
sumire < apk info -P mandoc
mandoc-1.14.6-r4 provides:
cmd:man=1.14.6-r4
sumire < apk info -P man-db
man-db-2.10.2-r...currently, the default cmd: providers have a version appended. so, for instance, both mandoc and man-db provide:
```
sumire < apk info -P mandoc
mandoc-1.14.6-r4 provides:
cmd:man=1.14.6-r4
sumire < apk info -P man-db
man-db-2.10.2-r1 provides:
cmd:man=2.10.2-r1
```
this then means that adding a provider_priority to each is worthless- if something depends on `cmd:man`, they will always pull in man-db, because it has a bigger version, and the priority is ignored entirely, because the priority is only used for things without a version.
this goes for a lot of uses of provider_priority= , as it's only useful if you make your own provides="randomstuffhere" virtual, as otherwise the embedded versions will make it failhttps://gitlab.alpinelinux.org/alpine/abuild/-/issues/10070checkapk soname diff does not work with busybox2023-06-26T01:27:26ZGhost Usercheckapk soname diff does not work with busyboxthe check is currently:
```sh
diff "../filelist-$_pkgname-old" "../filelist-$_pkgname" | awk '/>.*\.so/{$1 = ""; print $0}' | while read i; do
echo "${i}: " "$(objdump -p "$i" | grep SONAME)"
done
```
this assumes the diff starts wi...the check is currently:
```sh
diff "../filelist-$_pkgname-old" "../filelist-$_pkgname" | awk '/>.*\.so/{$1 = ""; print $0}' | while read i; do
echo "${i}: " "$(objdump -p "$i" | grep SONAME)"
done
```
this assumes the diff starts with '>', which is not supported in busybox diff, as it supports unified diffs only.
to work with both, diff -U3 should be used, and the awk should be amendedhttps://gitlab.alpinelinux.org/alpine/abuild/-/issues/10059Allow extra text after maintainer comment2022-01-22T11:27:07ZLuca WeissAllow extra text after maintainer commentIt would be useful to allow e.g. the following
```
# Maintainer: Foo Bar <foo@bar.com> @foobar
```
where `@foobar` is the GitLab username of this person.
It can be quite complicated from looking at an APKBUILD to find the correct perso...It would be useful to allow e.g. the following
```
# Maintainer: Foo Bar <foo@bar.com> @foobar
```
where `@foobar` is the GitLab username of this person.
It can be quite complicated from looking at an APKBUILD to find the correct person to tag in GitLab so this would allow this to be simplified.
This is less relevant for Alpine Linux because of the automatic tagging by algitbot (because it can do email->username lookup) but in postmarketOS this is not possible as far as I'm aware as we're on gitlab.com and don't have access to this lookup.https://gitlab.alpinelinux.org/alpine/abuild/-/issues/10055Broken package for empty package with post-install when built on btrfs2023-05-26T12:35:36ZLuca WeissBroken package for empty package with post-install when built on btrfsAPKBUILD:
```
pkgname=foobar
pkgver=1
pkgrel=0
pkgdesc="Foobar"
url="https://example.org/"
arch="all"
license="WTFPL"
install="$pkgname.post-install"
options="!check"
package() {
mkdir -p "$pkgdir"
}
sha512sums=""
```
foobar.post...APKBUILD:
```
pkgname=foobar
pkgver=1
pkgrel=0
pkgdesc="Foobar"
url="https://example.org/"
arch="all"
license="WTFPL"
install="$pkgname.post-install"
options="!check"
package() {
mkdir -p "$pkgdir"
}
sha512sums=""
```
foobar.post-install:
```
#!/bin/sh
echo "foobar!"
```
Build this on a btrfs partition and the resulting `.PKGINFO` in the apk will have `size = 0` set instead of `size = 4096` when built on ext4. This causes apk to classsify this as virtual package and not run the post-install.
A similar issue was fixed with https://gitlab.alpinelinux.org/alpine/abuild/-/commit/29557a4a5432e084207ae7f746f50be4cfd9c070 but as this package doesn't install any files this if doesn't apply. It only contains a post-install which gets added to the directory later in the build process.https://gitlab.alpinelinux.org/alpine/abuild/-/issues/10054Run rootbld on memory?2022-07-24T21:37:02ZdonobanRun rootbld on memory?I think that in case of having plenty of unused memory it's probably better idea use it instead the hard disk for something like rootbld.
Since I mount /tmp as tmpfs, I thought that doing a little workaround on 'abuild' would be enough,...I think that in case of having plenty of unused memory it's probably better idea use it instead the hard disk for something like rootbld.
Since I mount /tmp as tmpfs, I thought that doing a little workaround on 'abuild' would be enough, but then 'abuild-rmtemp' complains because it is not a valid path.
Altought this can achevieded with little customization I wonder if it could be an integrated option. In that case, instead of require a tmpfs mountpoint already available on the system, it could mount the '/var/tmp/abuild.XXXXiHMEOI/' as a tmpfs itself.https://gitlab.alpinelinux.org/alpine/abuild/-/issues/10046sbsigntool: segfaults using openssl-3.0.02021-10-07T10:24:15ZCássiosbsigntool: segfaults using openssl-3.0.0After updating openssl to 3.0.0 sbsign segfaults when trying to sign a EUFI image, breaking secureboot-hook, possibly making the system unbootable.After updating openssl to 3.0.0 sbsign segfaults when trying to sign a EUFI image, breaking secureboot-hook, possibly making the system unbootable.