GitLab

For riscv64, we temporarily need to be able to use a custom mirror. The
mirror is already a variable, but make sure we can override it with an
environment variable.

`abuild-keygen -i` uses `cp -i` to copy the public key to prevent it
from being unexpectedly being overwritten. This either causes the test
to hang, because it waits for stdin, or to fail, when no stdin is
available, which happened occasionally.

Patch abuild-keygen to ommit '-i' to prevent this issue.

With the change of pipestatus to pipefail, we forgot to capture stderr
as well, which was captured with pipestatus.

Even though POSIX sh does not have pipefail, busybox ash does have it.
This lets us remove the pipestatus library, which, even though it works,
causes flaky behavior in our test-suite in CI.

The search pattern was pretty lax, resulting in unintended changes. New
settings have been added to abuild.conf that were mangled:

> export CARGO_BUILD_JOBS=$JOBS

became

> export CARGO_BUILD_JOBS=$(nproc)$JOBS

This in turn results in CI jobs for rust packages timing out on
dowloading crates.

Fix this by being more strict in what is matched.

Fixes alpine/infra/infra#10721

When no packages are built, `~/packages/` is empty, resulting into an
error when referring to `~/packages/*`.

Just ignore any errors for now.

See merge request !6

With `set -u`, the shell will return an error. Use `${-}` to provide an
empty default value.

When piping commands, the exit code of all commands except the last one
is swallowed. This is problematic, because it prevents us properly
detecting the if the build is successful, because we want to be able to
pipe the output of commands to capture the output.

POSIX sh however has no way to determine the exit status of earlier
commands in the pipe.

[pipestatus][0] is a small single-file library that has some functions
that emulate a pipeline, and provide the exit status of each command.
There is also a convenient function `runpipe0`, that will return only 0
if all the commands in the pipeline ran successful.

[0]:https://github.com/cheusov/pipestatus/

In order to make it easier to extract information from the buildlogs,
write certain parts of the buildlogs to dedicated report logs.

These logs can then be expose through artifacts.

The maximum size of artifacts that can be uploaded to gitlab is limited
to reduce the storage requirements. When packags are built successfully,
but the combined package size is too large, the CI job will still fail,
because it failed to upload the artifacts.

To prevent this from happening, only offer the packages for upload as
artifacts when the combined size is lower than a set limit. This will
allow the CI job to succeed at the cost of not being able to download
the built packages as artifacts.

This also introduces a test suite to test the the build script.

To allow them to be uploaded as artifacts, they need to be available in
the project directory. The files are now copied in the CI script, but
the issue is that when the files are too large, the CI job fails.

This is solved by copying these files only when they are smaller than
the limit.

This adds a basic test based on bats to verify that the build script
works. This can be expanded with more tests to cover all functionality
and edge cases.

This makes it easier to test the script by allowing to specify a
REPODEST in a temporary location.

This variable is however not used consistently, so adjust the places
that refer directly to `$HOME/packages` to use `$REPODEST`.

If the tag already exists (because the repo is reused), then git will
fail to create the tag by default.

Shellcheck wants you to quote all variables to prevent accidental
wordsplitting. But the arguments to `ap builddirs` should be splitted
because each package should be a separate argument.

23ae6625 (build.sh: fix shellcheck issues, 2020-02-03) did quote
`$aports`, breaking CI when trying to build multiple packages.

Undo that change and add comments to show that it should remain unquoted.

See merge request !5

Only when CI_DEBUG_BUILD is non-empty

Shellcheck flagged various issues, like quoting variables to prevent
splitting. Let's fix those.

Closes alpine/infra/infra#10670

See merge request !4

The testing repo is only available for edge, so do not add it to the
repositories file when building for stable releases.

Fixes: alpine/infra/infra#10670

Setting up a base build setup has been moved to a dedicated image called
alpinelinux/build-base. This allows that functionality be reused and
makes setting up this image simpler.

Because `set -u` is configured, the script will give an error when
trying to use a variable that is not defined.

Because the purpose this is meant to be an environment variable, we
cannot just set it empty, because it would overwrite the value from the
environment.

Define `CI_DEBUB_BUILD` as empty when it's not defined.

Only when CI_DEBUG_BUILD is non-empty, verbose logging will be enabled.

To make it easer to find out what's going on, print the commit graph of
the repo.

Merge requests can consist of packages from different repositories. Some
packages will depend on the packages that are just being built in a
higher repository.

The script currently only adds repos from the mirror, which will cause
build issues.

Add the local higher repositories to the repositories list.

A plain git diff will return changes both from the branch being merged,
as the target branch. Because we are only interested in changes from the
current branch, we need to do an asymmetric diff.

To make it clear in the build logs what is being built, provide the list
of changed aports that the build script is trying to build.

The CI job for merge requests is run in the forked project, meaning that
the target branch may be behind. This would result in more commits being
checked than intended.

Fetch the target branch from the upstream repository instead so that we
know the target branch is up-to-date and we can reliably get a list of
changed commits just for this merge request.

The basebranch might already exist. When trying to update the basebranch
when it's not fast-forwardable, git will throw an error.

Add a '+' to the fetch spec to tell git that it's allowed to force
overwrite the current ref.