Skip to content
Snippets Groups Projects
  1. Apr 13, 2022
    • Kevin Daudt's avatar
      build: chown repository as current user · ac5a7e60
      Kevin Daudt authored
      As a security fix, git no longer allows executing in repositories owned
      by a different user. gitlab-runner clones the repositories as root,
      while the build script runs as a non-root user, resulting git in
      erroring out.
      
      As a simple fix, chown all files in the repository as the current user.
      ac5a7e60
  2. Mar 02, 2022
    • Kevin Daudt's avatar
      merge: functions: apply diff-filter to changed_repos · 4bfe7add
      Kevin Daudt authored
      `changed_aports` uses `--diff-filter=ACMR` so that it only get
      aports with relevant changes. Removed aports for example do not need to
      be built.
      
      But `changed_repos` omitted this filter. This lead to discrepancies
      between these two functions. This specifically resulted into issues when
      packages were moved.
      
      `changed_repos` would for example return 'main', when a package would be
      moved from main to community. `changed_aports` then would not report any
      package to be changed. But the way the amount of changed packages are
      counted, it would report '1', even though no packages have been built
      for this repo.
      
      This in turn results `build_start` going negative, which is used as a
      start range for `sed`, which then tries to interpret it as an argument
      which fails.
      
      Fix this by applying the same filter to `changed_repos` as
      `changed_aports` uses.
      
      See: alpine/infra/docker/alpine-gitlab-ci!11
      4bfe7add
    • Kevin Daudt's avatar
      functions: apply diff-filter to changed_repos · 95898ca4
      Kevin Daudt authored
      `changed_aports` uses `--diff-filter=ACMR` so that it only get
      aports with relevant changes. Removed aports for example do not need to
      be built.
      
      But `changed_repos` omitted this filter. This lead to discrepancies
      between these two functions. This specifically resulted into issues when
      packages were moved.
      
      `changed_repos` would for example return 'main', when a package would be
      moved from main to community. `changed_aports` then would not report any
      package to be changed. But the way the amount of changed packages are
      counted, it would report '1', even though no packages have been built
      for this repo.
      
      This in turn results `build_start` going negative, which is used as a
      start range for `sed`, which then tries to interpret it as an argument
      which fails.
      
      Fix this by applying the same filter to `changed_repos` as
      `changed_aports` uses.
      95898ca4
  3. Jan 16, 2022
    • Kevin Daudt's avatar
      build: restore ARCH variable · 7474477d
      Kevin Daudt authored
      It is still used.
      v20220302
      7474477d
    • Kevin Daudt's avatar
    • Kevin Daudt's avatar
      build: add sections for log output · c4235a8f
      Kevin Daudt authored
      gitlab supports section markers which can be used specify collapsible
      sections to make the build log better readable.
      c4235a8f
    • Kevin Daudt's avatar
      build: fix warning about REPODEST being read-only · b2b884d5
      Kevin Daudt authored
      `REPODEST` was declared read-only in build.sh. Later, 'abuild.conf',
      which also defines `REPODEST`, is being sourced in a subshell. This
      resulted in a warning about `REPODEST` being read-only.
      
      Fix this by defining REPODEST as a normal variable, not a read-only
      variable.
      b2b884d5
    • Kevin Daudt's avatar
      build: simplify changed_aports · 1ac30a9a
      Kevin Daudt authored
      changed_aports pipes the result to `xargs` to make it space delimited
      like the output of `ap builddirs`, for the consumer to pipe it to `xargs
      -n1 echo` to make it line delimited again for sed.
      
      Remove these redundant steps.
      1ac30a9a
    • Kevin Daudt's avatar
      git: set default branch name · 159d7067
      Kevin Daudt authored
      This suppresses the hint that happens in every CI build.
      159d7067
    • Kevin Daudt's avatar
      build: extract common functions to a library · 01e7d175
      Kevin Daudt authored
      This way the functionality can be reused
      01e7d175
    • Kevin Daudt's avatar
      tests: remove abuild-keygen interactive cp workound · da688cf5
      Kevin Daudt authored
      This has been fixed in abuild-keygen, so we no longer need to work
      around it.
      da688cf5
    • Kevin Daudt's avatar
    • Kevin Daudt's avatar
      build: fix build when repos do not have changes · 95b5cd6e
      Kevin Daudt authored
      The `changed_repos` function determines what repos have changes so that
      it only builds repos that actually have changes. But this function just does
      a plain diff against the target branch. This means if the target branch
      has any changes compared to the source branch, it would consider that
      branch as well.
      
      This was never a real issue, because the `changed_aports` function
      correctly returns the changed aports for a give repo, which would be
      empty in case there are no changed aports for that repo. This is also no
      a problem in general, because the other repositories always have aports.
      This would break when a repository does not have any aports, because `ap
      builddirs` would throw an error.
      
      New functionality has been added that determines how many aports have
      changed and tries to select only a subset. To do that, it needs to
      convert a space delimited list into a new-line delimited list. It uses
      `xargs -n1 echo` for that. The problem is that by default, `xargs` runs
      echo as well even if there are no items passed, resulting in a single
      newline being returned. Passing that to `wc -l` returns '1' while no
      aport was changed. This throws off the accounting for limiting the
      packages being built.
      
      Fix this by both properly doing an asymmetric diff against the target
      branch in `changed_repos` and by adding `-r` to `xargs` so that it
      returns nothing when there are no changed aports.
      95b5cd6e
  4. Jan 15, 2022
  5. Jan 13, 2022
    • Kevin Daudt's avatar
      build: support building a subset of packages · 8e742333
      Kevin Daudt authored
      This adds 2 environment variable to control what subset of the changed
      packages to build.
      
      The offset determines where to start in the list of changed packages,
      and the limit determines how many to build.
      8e742333
  6. Dec 06, 2021
  7. Nov 28, 2021
  8. Oct 19, 2021
  9. Jun 17, 2021
  10. Jun 14, 2021
    • Kevin Daudt's avatar
      tests: fix occasional hangs during test runs · 129272b5
      Kevin Daudt authored
      `abuild-keygen -i` uses `cp -i` to copy the public key to prevent it
      from being unexpectedly being overwritten. This either causes the test
      to hang, because it waits for stdin, or to fail, when no stdin is
      available, which happened occasionally.
      
      Patch abuild-keygen to ommit '-i' to prevent this issue.
      129272b5
    • Kevin Daudt's avatar
      build: capture stderr as well · 565be83e
      Kevin Daudt authored
      With the change of pipestatus to pipefail, we forgot to capture stderr
      as well, which was captured with pipestatus.
      565be83e
    • Kevin Daudt's avatar
      build: use pipefail · 5d90b52e
      Kevin Daudt authored
      Even though POSIX sh does not have pipefail, busybox ash does have it.
      This lets us remove the pipestatus library, which, even though it works,
      causes flaky behavior in our test-suite in CI.
      5d90b52e
  11. May 19, 2021
    • Kevin Daudt's avatar
      build: restrict JOBS adjustment · 99cdded8
      Kevin Daudt authored
      The search pattern was pretty lax, resulting in unintended changes. New
      settings have been added to abuild.conf that were mangled:
      
      > export CARGO_BUILD_JOBS=$JOBS
      
      became
      
      > export CARGO_BUILD_JOBS=$(nproc)$JOBS
      
      This in turn results in CI jobs for rust packages timing out on
      dowloading crates.
      
      Fix this by being more strict in what is matched.
      
      Fixes alpine/infra/infra#10721
      99cdded8
  12. Mar 07, 2021
  13. Feb 27, 2021
  14. Feb 26, 2021
  15. Feb 22, 2021
  16. Feb 21, 2021
    • Kevin Daudt's avatar
      merge: limit the size of uploaded packages · 5686ce18
      Kevin Daudt authored
      The maximum size of artifacts that can be uploaded to gitlab is limited
      to reduce the storage requirements. When packags are built successfully,
      but the combined package size is too large, the CI job will still fail,
      because it failed to upload the artifacts.
      
      To prevent this from happening, only offer the packages for upload as
      artifacts when the combined size is lower than a set limit. This will
      allow the CI job to succeed at the cost of not being able to download
      the built packages as artifacts.
      
      This also introduces a test suite to test the the build script.
      5686ce18
  17. Feb 20, 2021
    • Kevin Daudt's avatar
      build: copy packages after build · c11f3f45
      Kevin Daudt authored
      To allow them to be uploaded as artifacts, they need to be available in
      the project directory. The files are now copied in the CI script, but
      the issue is that when the files are too large, the CI job fails.
      
      This is solved by copying these files only when they are smaller than
      the limit.
      c11f3f45
    • Kevin Daudt's avatar
      ci: add test suite to verify functionality · aa67ec05
      Kevin Daudt authored
      This adds a basic test based on bats to verify that the build script
      works. This can be expanded with more tests to cover all functionality
      and edge cases.
      aa67ec05
    • Kevin Daudt's avatar
      build: allow override of REPODEST · 7a65c4ee
      Kevin Daudt authored
      This makes it easier to test the script by allowing to specify a
      REPODEST in a temporary location.
      
      This variable is however not used consistently, so adjust the places
      that refer directly to `$HOME/packages` to use `$REPODEST`.
      7a65c4ee
  18. Feb 10, 2020
  19. Feb 09, 2020
Loading