1. 25 Sep, 2021 5 commits
    • Kevin Daudt's avatar
      ci: also run publish job for stable branches · 93637bd2
      Kevin Daudt authored
      93637bd2
    • Kevin Daudt's avatar
      merge: split off gitaly, gitlab-shell and ssh · b7e7018b
      Kevin Daudt authored
      Except for postgres and redis, the image is a monolithic image. This made it
      easier to build the image, because all components are linked to each-other. This
      MR starts the process of splitting out components, namely gitaly, and related,
      gitlab-shell together with openssh.
      
      To make sure each component has the correct version, a script located at
      `build/versions.sh` will obtain the right versions of gitaly and gitlab-shell
      from gitlab. There is a [task file](https://taskfile.dev) that automates
      building of the docker images. One of the benefits is that it provides a way
      to use any file as an env file, something that docker-compose lacks. Only a the
      `.env` file it sources for environment variables that are usable as build arguments
      for docker images. Environment variables located in files specified by `env_file`
      directives are only usable as run-time variables for containers. So we can either
      use `task`, or we need to concatenate the build version environment variables to
      `.env`.
      
      The gitaly and gitlab-shell images are built using docker multi-stage builds. This
      makes it easier to keep the final images clean, but also allows for parallel
      building of each stage. This can be achieved by using docker buildkit by setting
      `DOCKER_BUILDKIT=1`. By making sure each stage has dedicated files for building,
      changing files for one stage should not cause other stages to be rebuilt.
      
      The different components communicate with each other through sockets, so volumes
      are used to mount the directories containing these sockets in the different images.
      `gitlab-shell` at the moment is used in both gitlab, gitally, was a container for
      openssh, so the files are copied from the gitlab-shell container into gitaly and
      gitlab-shell. In the future, this might no longer be necessary.
      
      This will also switch building and publishing the images from drone ci to gitlab-ci.
      The reason this was not done yet, is that you would create a dependency on gitlab to
      build gitlab. If we would publish a version that is completely broken, then we have no
      image to fall back on again. This is addressed by using versioned tags, so that it's
      easier to switch back to a working image.
      
      See merge request !8
      b7e7018b
    • Kevin Daudt's avatar
      task: generate .env files · 474e5388
      Kevin Daudt authored
      The .env file is the only way docer-compose supports loading variables
      that are globally available in the .docker-compose file. This file is
      not committed because it's also used to inject secrets.
      
      There are some variables that we need to keep track in the repository
      because they contain all the different versions of components that need
      to be used.
      
      Sadly this results into more and more reliance on something like
      Taskfile, but we're running into the limitations of docker-compose.
      474e5388
    • Kevin Daudt's avatar
      gitlab: don't publish port 80 · 04065b60
      Kevin Daudt authored
      We use traefik now to serve http, so we no longer need to publish port 80
      04065b60
    • Kevin Daudt's avatar
      ci: add job to upload images to docker hub · b7599781
      Kevin Daudt authored
      There is no longer a single image to upload, but multiple images. This
      is a good time to switch over from drone-ci to our own CI.
      
      The images will be tagged, making it easier to switch back to an earlier
      version.
      b7599781
  2. 24 Sep, 2021 9 commits
  3. 07 Aug, 2021 3 commits
    • Kevin Daudt's avatar
      gitlab-shell: extract separate image · b30611f5
      Kevin Daudt authored
      gitaly has a dependency on gitlab-shell, so we build an image that
      builds gitlab-shell which gitaly use to copy the gitlab-shell binaries
      from.
      
      We create a separate image instead of a separate build stage in gitaly
      because the gitlab-shell image will be used to run openssh.
      b30611f5
    • Kevin Daudt's avatar
      gitaly: finalize image to make gitaly run · f22b1713
      Kevin Daudt authored
      Make sure expected files and directories are in place, and provide a
      entrypoint that will start gitaly.
      f22b1713
    • Kevin Daudt's avatar
      gitaly: split up build script · 94c66097
      Kevin Daudt authored
      With a single script that is copied in each stage, any changes forces
      all build stages to be rebuilt.
      
      Instead, split up the script in separate scripts that each git copied
      just to their respective stages. The same counts for the patches.
      
      Some functionality is shared by different build stages. We extract this
      functionality in a separate lib file that can be sourced between stages
      and even in different components.
      94c66097
  4. 13 Jul, 2021 5 commits
    • Kevin Daudt's avatar
      docker/gitaly: build final image · 6aa9c43b
      Kevin Daudt authored
      This copies the build artifacts from earlier images to compose the final
      image. To make sure we have all soname dependencies, use scanelf to find
      the sonames needed for each binary so that they can be installed.
      
      This is easier than manually tracking dependencies and is based on the
      gemdeps.so script that is used in the gitlab container.
      6aa9c43b
    • Kevin Daudt's avatar
      docker/gitaly: add stage for ruby components · 141cfec2
      Kevin Daudt authored
      Builds the ruby components of gitaly that has been skipped before.
      141cfec2
    • Kevin Daudt's avatar
      docker/gitaly: add a stage for git · 4f6a2157
      Kevin Daudt authored
      This makes sure that the version of git matches what gitaly expects, and
      gitaly has optional patches that it applies when building git.
      4f6a2157
    • Kevin Daudt's avatar
      docker/gitaly: add stage for gitaly · 9b1eafa0
      Kevin Daudt authored
      This builds the go components of gitaly. To prevent the make file from
      also building the ruby component, the Makefile is patched to remove the
      dependency. The ruby component will be built in a separate stage.
      
      To reduce the size of the files that end up in the final image, we strip
      the binaries of debug symbols using `strip`.
      9b1eafa0
    • Kevin Daudt's avatar
      docker/gitaly: add source stage · d7061593
      Kevin Daudt authored
      This stage will download the gitaly source so that it can be used in
      other stages without having to redownload it each time.
      d7061593
  5. 12 Jul, 2021 1 commit
    • Kevin Daudt's avatar
      build: generate env file with component versions · bd57bc28
      Kevin Daudt authored
      When splitting up gitlab into components, we need to make sure that we
      get the correct versions of all the components. These versions are
      defined in the gitlab source as separate files.
      
      We use the gitlab API to get the contents of the file and write an env
      file with the version of each component. This can be made available as
      build arguments to the Dockerfiles.
      
      This uses go-task, which is a simple and straightforward way to define
      common tasks.
      bd57bc28
  6. 10 Jul, 2021 1 commit
  7. 09 Jul, 2021 3 commits
  8. 08 Jul, 2021 1 commit
  9. 02 Jul, 2021 1 commit
    • Kevin Daudt's avatar
      setup: install git from source · a3c26ab7
      Kevin Daudt authored
      Gitaly requires git v2.31.0 or higher, but alpine:3.13 only has v2.30.0.
      Additionally, gitally provides a build option for git from source where
      they include additional patches.
      
      So use the git version that is built by gitaly instead of git shipped by
      Alpine Linux.
      a3c26ab7
  10. 01 Jul, 2021 3 commits
  11. 02 May, 2021 1 commit
  12. 15 Apr, 2021 1 commit
  13. 14 Apr, 2021 1 commit
  14. 10 Apr, 2021 1 commit
  15. 17 Mar, 2021 1 commit
  16. 04 Mar, 2021 1 commit
  17. 15 Feb, 2021 1 commit
  18. 07 Feb, 2021 1 commit