apk-tools issueshttps://gitlab.alpinelinux.org/alpine/apk-tools/-/issues2023-12-19T12:23:07Zhttps://gitlab.alpinelinux.org/alpine/apk-tools/-/issues/10819manage user and group creation/deletion directly2023-12-19T12:23:07ZAriadne Conillariadne@ariadne.spacemanage user and group creation/deletion directlyPresently, the majority of pre-install/post-install scripts in Alpine are doing user/group management. If apk did this instead, we could drop these scripts in the future.Presently, the majority of pre-install/post-install scripts in Alpine are doing user/group management. If apk did this instead, we could drop these scripts in the future.v3.1https://gitlab.alpinelinux.org/alpine/apk-tools/-/issues/10829Understanding .apk-new protected files behavior2023-12-02T13:31:07ZYann BizeulUnderstanding .apk-new protected files behaviorMy projects is made of several scripts and docker compose file, installed in `/usr/local/myproject/`.
The user can alter docker-compose behavior by editing a `/usr/local/myproject/docker-compose.override.yaml` file that is installed by ...My projects is made of several scripts and docker compose file, installed in `/usr/local/myproject/`.
The user can alter docker-compose behavior by editing a `/usr/local/myproject/docker-compose.override.yaml` file that is installed by default with pretty much everything commented out, they can go in and add their own configuration.
The problem is that this file gets overwritten with a package update, my assumption after doing some research is that apk doesn't consider this file a being _protected_ and assumes it has to be overwritten with the package version.
What I would like, is instead to replace the file if it has been untouched, or if it has, copy the distribution file as `.apk-new` which seems to be the standard behavior regarding configuration files.
I'm thinking the path I use isn't considered as appropriate for APK to adopt protected mode.
- Is there a way to indicate APK that a given file is a configuration file and shouldn't be overwritten ?
- Where can I find more information about that feature ? (does it look at specific folders, specific file extensions ?)
Thanks for your help.https://gitlab.alpinelinux.org/alpine/apk-tools/-/issues/10832CI tests on macOS2022-12-21T19:39:45ZPaul SpoorenCI tests on macOSAs a follow up of https://gitlab.alpinelinux.org/alpine/apk-tools/-/issues/10794 it would be nice to add CI tests running on macOS.As a follow up of https://gitlab.alpinelinux.org/alpine/apk-tools/-/issues/10794 it would be nice to add CI tests running on macOS.backloghttps://gitlab.alpinelinux.org/alpine/apk-tools/-/issues/10842retain signatures of installed packages in apkdb2022-12-21T18:50:51ZAriadne Conillariadne@ariadne.spaceretain signatures of installed packages in apkdbA current problem with security scanners are that they just check the version field and assume all packages are coming from the same ecosystem.
This means that, as an attacker, I can build an old version (say nginx `1.20.2-r0`) of a pac...A current problem with security scanners are that they just check the version field and assume all packages are coming from the same ecosystem.
This means that, as an attacker, I can build an old version (say nginx `1.20.2-r0`) of a package which has vulnerabilities, and distribute it through an alternative repository, while saying it is the latest version (nginx `1.20.2-r9000`), and the security scanners will say this is entirely fine, because the version is higher than the last vulnerable one.
We already have a useful piece of data that can be used to correlate whether or not a package legitimately comes from the distribution ecosystem: the package signature. If we retain it, then scanners can check the signature to determine which ecosystem (and thus, security feeds) the package should be checked against (or it can warn that the package does not come from an ecosystem supported by that scanner).v3.1https://gitlab.alpinelinux.org/alpine/apk-tools/-/issues/10844solver tries to swap packages when presented with a multi-arch index2022-12-21T19:39:44ZAriadne Conillariadne@ariadne.spacesolver tries to swap packages when presented with a multi-arch indexDue to a bug we recently created an index which had some packages for `aarch64` and other packages for `x86_64`.
When we tried to install packages using this repo, it tried to swap the dependencies for the `aarch64` packages that were i...Due to a bug we recently created an index which had some packages for `aarch64` and other packages for `x86_64`.
When we tried to install packages using this repo, it tried to swap the dependencies for the `aarch64` packages that were in the repo, even though they were already installed as `x86_64` packages.backloghttps://gitlab.alpinelinux.org/alpine/apk-tools/-/issues/10845implement support for SCT signature scheme in apkv32022-12-21T18:56:12ZAriadne Conillariadne@ariadne.spaceimplement support for SCT signature scheme in apkv3[Fulcio](https://github.com/sigstore/fulcio) is a reference implementation of the SCT signature scheme, which allows for signing keys to be created on demand and retired after signing an artifact.
In our downstream distribution, we pres...[Fulcio](https://github.com/sigstore/fulcio) is a reference implementation of the SCT signature scheme, which allows for signing keys to be created on demand and retired after signing an artifact.
In our downstream distribution, we presently generate APKv2 files with signatures like `.SIGN.FULCIO.[OIDC-identity-hash]` and validate their signatures when including them in our repositories. This works fine because of the APKv2 trust model.
However, APKv3 does not allow us to just "invent" a new signature scheme and use it for our own purposes. Accordingly we would like to add a signature type "fulcio" which stores the SCT signature data, but otherwise uses the APKv2 trust model (for now).
Thoughts?v3.1https://gitlab.alpinelinux.org/alpine/apk-tools/-/issues/10846Multiple maintainers in metadata?2022-12-21T18:53:53ZPaul SpoorenMultiple maintainers in metadata?Is it possible to store multiple maintainers in the package metadata or is it always a single string only?Is it possible to store multiple maintainers in the package metadata or is it always a single string only?v3.0https://gitlab.alpinelinux.org/alpine/apk-tools/-/issues/10851Support files changing owner in the same transaction without replaces=?2022-12-21T18:56:12ZOliver SmithSupport files changing owner in the same transaction without replaces=?During an upgrade, sometimes a new-package gets installed instead of an old-package, and takes over files from old-package. If that is the case, the new-package must have `replaces="old-package"` or else we get an error like the followin...During an upgrade, sometimes a new-package gets installed instead of an old-package, and takes over files from old-package. If that is the case, the new-package must have `replaces="old-package"` or else we get an error like the following:
```
(491/542) Installing bemenu (0.6.7-r0)
ERROR: bemenu-0.6.7-r0: trying to overwrite usr/bin/bemenu owned by sxmo-bemenu-0.6.3.0-r0.
```
(example from [aports!34975](https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/34975))
In other words:
* old
* a package depends on "sxmo-bemenu"
* new
* the same package does not depend on "sxmo-bemenu" anymore, but instead on "bemenu"
* bemenu owns the files now
Since it's in the same transaction, and running `apk fix` afterwards will resolve it, I wonder if apk could just automatically recognize this during the upgrade and not print an error even if there is no `replaces=` present.v3.1https://gitlab.alpinelinux.org/alpine/apk-tools/-/issues/10856apk3: failure upgrading packages if newer version replaces file with directory2022-12-21T18:56:12ZDaniel Kolesaapk3: failure upgrading packages if newer version replaces file with directoryOne will get a `not a directory` error when trying to upgrade a package and the newer version has a directory where the previous version had a file.One will get a `not a directory` error when trying to upgrade a package and the newer version has a directory where the previous version had a file.v3.1https://gitlab.alpinelinux.org/alpine/apk-tools/-/issues/10858`apk add ...` should attempt to redownload an apk package with a bad signatur...2022-12-21T18:56:11ZEllie`apk add ...` should attempt to redownload an apk package with a bad signature at least once during the command run`apk add ...` should attempt to redownload an apk package with a bad signature at least once during the command run. Instead it just gets stuck dumbfounded and always gives the same failed output even if the command is rerun:
```
$ sudo...`apk add ...` should attempt to redownload an apk package with a bad signature at least once during the command run. Instead it just gets stuck dumbfounded and always gives the same failed output even if the command is rerun:
```
$ sudo apk add py3-pip
(1/12) Installing py3-contextlib2 (21.6.0-r2)
(2/12) Installing py3-tomli (2.0.1-r1)
(3/12) Installing py3-pep517 (0.12.0-r2)
(4/12) Installing py3-six (1.16.0-r1)
(5/12) Installing py3-retrying (1.3.3-r3)
2% ██
(6/12) Installing py3-appdirs (1.4.4-r3)
2% ██
3% ██
(7/12) Installing py3-more-itertools (8.13.0-r0)
3% ██
4% ███
5% ███
(8/12) Installing py3-ordered-set (4.0.2-r3)
5% ████
(9/12) Installing py3-parsing (2.4.7-r3)
5% ████
ERROR: py3-parsing-2.4.7-r3: IO ERROR
(10/12) Installing py3-packaging (21.3-r0)
8% ██████
ERROR: py3-packaging-21.3-r0: BAD signature
(11/12) Installing py3-setuptools (59.4.0-r0)
10% ███████
ERROR: py3-setuptools-59.4.0-r0: BAD signature
(12/12) Installing py3-pip (22.1.1-r0)
23% █████████████████
ERROR: py3-pip-22.1.1-r0: BAD signature
100% █████████████████████████████████████4 errors; 2154 MiB in 814 packages
$
```
For such an obvious culprit, I would expect it to do *something* when it runs into broken packages like that. at the very least when I try running the command again.
I saw this with apk 2.12.9 on postmarketOS 22.06.v3.1https://gitlab.alpinelinux.org/alpine/apk-tools/-/issues/10861implement support for OCI registries2022-12-21T18:56:11ZAriadne Conillariadne@ariadne.spaceimplement support for OCI registriesWith projects such as ORAS (OCI Registry as Storage), it may be interesting to support fetching apk packages as if they were objects stored in OCI registries. This would allow users to distribute apk packages at zero cost by using pre-e...With projects such as ORAS (OCI Registry as Storage), it may be interesting to support fetching apk packages as if they were objects stored in OCI registries. This would allow users to distribute apk packages at zero cost by using pre-existing OCI registries such as DockerHub, GHCR, etc.
Other package managers, such as Homebrew, have implemented support for this distribution method. I believe OpenWrt developers have expressed interest in this as well.
(This would be for something after apk 3.0 release.)v3.1https://gitlab.alpinelinux.org/alpine/apk-tools/-/issues/10862Fetch fails to detect broken IPv62022-12-21T19:39:44ZReuben LifshayFetch fails to detect broken IPv6If apk is running in a dual stack environment where AAAA records are returned but IPv6 packets do not go through fetch will try to connect only to the v6 addresses and will take a very long time to fail and fall back to v4. In my case, t...If apk is running in a dual stack environment where AAAA records are returned but IPv6 packets do not go through fetch will try to connect only to the v6 addresses and will take a very long time to fail and fall back to v4. In my case, this is with building docker images.
Other utilities such as `curl` work correctly in the same situation because they use the Happy Eyeballs algorithm ([RFC 8305](https://www.rfc-editor.org/rfc/rfc8305)).
apk:
```
~ # apk update
fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/main/x86_64/APKINDEX.tar.gz
...(hangs until timeout)
```
curl:
```
~ # curl -vI https://dl-cdn.alpinelinux.org/alpine/v3.16/main/x86_64/APKINDEX.tar.gz
* Trying 2a04:4e42:600::645:443...
* Trying 151.101.194.133:443...
* Connected to dl-cdn.alpinelinux.org (151.101.194.133) port 443 (#0)
* ALPN: offers h2
* ALPN: offers http/1.1
* CAfile: /etc/ssl/certs/ca-certificates.crt
* CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
...CLIPPED
```
curl IPv6 only:
```
~ # curl -6 -vI https://dl-cdn.alpinelinux.org/alpine/v3.16/main/x86_64/APKINDEX.tar.gz
* Trying 2a04:4e42::645:443...
...(hangs until timeout)
```backloghttps://gitlab.alpinelinux.org/alpine/apk-tools/-/issues/10865Add info on where each package in apk upgrade is coming from2022-12-21T18:56:12ZJustinAdd info on where each package in apk upgrade is coming fromIf the user has multiple repositories installed, show where each package is coming from.
For instance I have a script that does this with apt data and spits out this:
```
+------------------+-----------------+-------------------+------...If the user has multiple repositories installed, show where each package is coming from.
For instance I have a script that does this with apt data and spits out this:
```
+------------------+-----------------+-------------------+-------------------+
| Package Name | Repository | Current Version | New Version |
+------------------+-----------------+-------------------+-------------------+
| kpartx | focal-security | 0.8.3-1ubuntu2 | 0.8.3-1ubuntu2.1 |
| libexpat1 | focal-security | 2.2.9-1ubuntu0.4 | 2.2.9-1ubuntu0.5 |
| multipath-tools | focal-security | 0.8.3-1ubuntu2 | 0.8.3-1ubuntu2.1 |
+------------------+-----------------+-------------------+-------------------+
```v3.1https://gitlab.alpinelinux.org/alpine/apk-tools/-/issues/10867Documentation on cmd:PATTERN, so:PATTERN etc. missing from apk-search.82022-12-26T13:28:50ZPaul W. Rankinhello@paulwrankin.comDocumentation on cmd:PATTERN, so:PATTERN etc. missing from apk-search.8I'm not sure where I learnt that `apk-search(8)` could accept `cmd:PATTERN` but it seems it was not the man page.
Upon trying to learn which prefixes are available the documentation is missing.I'm not sure where I learnt that `apk-search(8)` could accept `cmd:PATTERN` but it seems it was not the man page.
Upon trying to learn which prefixes are available the documentation is missing.backloghttps://gitlab.alpinelinux.org/alpine/apk-tools/-/issues/10868add vendor field to apkv2/apkv3 control data and apkdb2024-03-08T14:45:39ZAriadne Conillariadne@ariadne.spaceadd vendor field to apkv2/apkv3 control data and apkdbSoftware component analysis tools (and related software such as vulnerability scanning tools) use CPE identifiers and Package URLs (PURLs) to determine the ecosystem a software component originates from.
At present, the apkdb does not c...Software component analysis tools (and related software such as vulnerability scanning tools) use CPE identifiers and Package URLs (PURLs) to determine the ecosystem a software component originates from.
At present, the apkdb does not contain this information, so SCA tools assume that all packages originate from the distribution. This causes problems when trying to understand the relationship between software components and their suppliers, to do things like find out about remediated CVEs, as the calculated CPE identifiers and PURLs are matched to the wrong supplier.
Since Alpine 3.16, we have shipped `/etc/secfixes.d/$vendorlabel`, e.g. `/etc/secfixes.d/alpine`, files in the baselayout. Some other apk distributions which use the same secdb format have done the same such as $dayjob's Wolfi GNU/Linux distribution. By providing a vendor tag in the APK control data and apkdb which matches to these `/etc/secfixes.d` files, we can correctly map relevant security remediation feeds to the packages.
I'm happy to work on this if @fabled agrees we should do this.v3.0https://gitlab.alpinelinux.org/alpine/apk-tools/-/issues/10870Cannot install packages with symlinks to FAT32 filesystems2023-01-16T17:14:39ZPatrycja Rosaalpine@ptrcnull.meCannot install packages with symlinks to FAT32 filesystemsin my specific case, having EFI system partition mounted at `/boot` and installing a package like `main/xen-hypervisor` fails on `symlinkat("xen-4.17.0.gz", 3, "boot/.apk...") = -1 EPERM` called from [here][1]
*technically* that's not a...in my specific case, having EFI system partition mounted at `/boot` and installing a package like `main/xen-hypervisor` fails on `symlinkat("xen-4.17.0.gz", 3, "boot/.apk...") = -1 EPERM` called from [here][1]
*technically* that's not an issue with apk itself - the package literally has a symlink and apk is just trying to do its job - but it would be nice if there was some kind of a workaround for it (or maybe a more helpful error message)
[1]: https://gitlab.alpinelinux.org/alpine/apk-tools/-/blob/v2.12.10/src/io_archive.c#L382https://gitlab.alpinelinux.org/alpine/apk-tools/-/issues/10873tooling for repository mirroring2023-04-11T17:55:40ZTimo Terästooling for repository mirroringImprove existing applets (e.g. `fetch`) or create new one(s) to allow for:
- downloading repository index and all packages in it
- support incremental download with modified-since timestamp (or possibly compared to an old index)
- rec...Improve existing applets (e.g. `fetch`) or create new one(s) to allow for:
- downloading repository index and all packages in it
- support incremental download with modified-since timestamp (or possibly compared to an old index)
- recreate repository index given index and list of directories where to look files from; create symlinks or hardlinks
- print list of package names present on indexv3.0Timo TeräsTimo Teräshttps://gitlab.alpinelinux.org/alpine/apk-tools/-/issues/10874conversion of installeddb and query applets to new database format2023-01-31T11:16:53ZTimo Teräsconversion of installeddb and query applets to new database formatConvert all internal code to use the new ADB based structures.
Support old indexes by doing on-the-fly conversion.
Automatically convert and keep installeddb in the new format.Convert all internal code to use the new ADB based structures.
Support old indexes by doing on-the-fly conversion.
Automatically convert and keep installeddb in the new format.v3.1Timo TeräsTimo Teräshttps://gitlab.alpinelinux.org/alpine/apk-tools/-/issues/10875add PURL field to apkv3 package data2024-03-08T14:45:39ZAriadne Conillariadne@ariadne.spaceadd PURL field to apkv3 package dataSecurity scanners have issues with disambiguating distribution package names (which exist in a flat namespace) verses upstream package names. In Alpine, a notable example of this would be the lua packages, e.g. `lua5.1`, `lua5.2`, etc. ...Security scanners have issues with disambiguating distribution package names (which exist in a flat namespace) verses upstream package names. In Alpine, a notable example of this would be the lua packages, e.g. `lua5.1`, `lua5.2`, etc. In these cases, scanners are unable to deduce that `lua5.1` is equivalent to upstream `lua~5.1` (e.g. lua 5.1.x).
An emergent industry standard to allow for disambiguation is the [Package URL specification](https://github.com/package-url/purl-spec). We should store this data in APKs, to help the scanners disambiguate.
(This would also require us to add PURLs where needed in our package build recipes, e.g. aports et al. But this is out of scope for here.)https://gitlab.alpinelinux.org/alpine/apk-tools/-/issues/10881slightly bizarre add/upgrade behaviour2023-03-01T12:56:16ZGhost Userslightly bizarre add/upgrade behaviouri don't have a good reproduction for this. what happens is:
- there is a prebuilt CI image that has some 'older' version of a package installed. in this case, the ncurses libraries.
- a CI run starts
- the image runs `apk update`. and t...i don't have a good reproduction for this. what happens is:
- there is a prebuilt CI image that has some 'older' version of a package installed. in this case, the ncurses libraries.
- a CI run starts
- the image runs `apk update`. and there are newer versions of the things already installed.
- the image then does `apk add` of a bunch of stuff that isn't already in `world`.
and it fails with:
```
ERROR: unable to select packages:
ncurses-terminfo-base-6.4_p20230218-r3:
breaks: libmenuw-6.4_p20230225-r0[ncurses-terminfo-base=6.4_p20230225-r0]
libformw-6.4_p20230225-r0[ncurses-terminfo-base=6.4_p20230225-r0]
satisfies: libpanelw-6.4_p20230218-r3[ncurses-terminfo-base=6.4_p20230218-r3]
libncursesw-6.4_p20230218-r3[ncurses-terminfo-base=6.4_p20230218-r3]
libncursesw-6.4_p20230218-r3:
breaks: ncurses-dev-6.4_p20230225-r0[libncursesw=6.4_p20230225-r0]
satisfies: python3-3.11.2-r0[so:libncursesw.so.6]
gettext-libs-0.21.1-r1[so:libncursesw.so.6]
libpanelw-6.4_p20230218-r3[so:libncursesw.so.6]
libmenuw-6.4_p20230225-r0[so:libncursesw.so.6]
pinentry-1.2.1-r0[so:libncursesw.so.6]
readline-8.2.001-r0[so:libncursesw.so.6]
libedit-20221030.3.1-r0[so:libncursesw.so.6]
libformw-6.4_p20230225-r0[so:libncursesw.so.6]
libpanelw-6.4_p20230218-r3:
breaks: ncurses-dev-6.4_p20230225-r0[libpanelw=6.4_p20230225-r0]
satisfies: python3-3.11.2-r0[so:libpanelw.so.6]
```
but, if the `apk add bunch of stuff` is preceded by `apk upgrade`, no errors happen. i don't know what causes this.
to be clear, i don't think there is anything invalid at all in the above packaging. the apkbuild for ncurses is [this](https://git.alpinelinux.org/aports/tree/main/ncurses/APKBUILD?id=0ad370efb53b5a388e82b62b21ec8733b34a7aad). strictly speaking, nothing actually conflicts here (between files/paths), upgrades work fine, etc. the failing CI job is https://gitlab.freedesktop.org/emersion/wlroots/-/jobs/37107011 .
once the CI image 'gets regenerated' so the already-installed packages are the same as the ones in the repositories, everything works fine. at that point, merely bumping the pkgrel with no changes on the above ncurses packages will get the same failure again, on starting the image and running `apk add some stuff`.
this /did/ work before. i think this is a regression in 2.12.11, but i can't point to anything because i don't have a good reproduction setup for this..
of course, merely running `upgrade` first fixes it, but as i understand it the mere `add` should also work?