GitLab CI/CD Series Appendix I: Custom GitLab Runner image to run anywhere

Even though GitLab offers free minutes on their CI/CD runners, it might not be enough for your project. Then there are two options:

pay GitLab and get additional minutes
setup your own runners

This post will focus on the second one. From my experience, the runners that GitLab offers for free can be extremely slow, especially when using cache in your CI/CD configuration. Even though paying GitLab seems much more worry-free, it's possible to have an easy and stressless way of setting up your own runners.

The runners can be easily run in a Docker container, and that's what this post will utilize. Let's start with a simple Docker Compose setup:

version: "3.9"

services:
  portainer:
    image: portainer/portainer-ce:latest
    container_name: portainer
    ports:
      - "8000:8000"
      - "9443:9443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - portainer_data:/data
    restart: always

volumes:
  portainer_data:

Right now, it contains one service seemingly unrelated to GitLab. And that's right, but Portainer is a great utility for managing containers, so it's good to include it (but not a must by any means). Another thing that will be useful is an object storage service. For this purpose, let's use MinIO, which is S3 compatible. That will play nicely with the GitLab runners.

version: "3.9"

services:
  minio:
    build:
      context: .
      dockerfile: minio.Dockerfile
    container_name: runner-cache
    ports:
      - "9000:9000"
      - "9001:9001"
    volumes:
      - minio_data:/data
    environment:
      MINIO_ROOT_USER: minio
      MINIO_ROOT_PASSWORD: minio123
    command: server --console-address ":9001" /data
    network_mode: bridge

  portainer:
    image: portainer/portainer-ce:latest
    container_name: portainer
    ports:
      - "8000:8000"
      - "9443:9443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - portainer_data:/data
    restart: always

volumes:
  portainer_data:
  minio_data:

docker-compose.yml

FROM minio/minio:latest
RUN mkdir -p /data/runner

minio.Dockerfile

The Dockerfile might seem funny, but it is the easiest way to ensure that the container where the cache should be stored will be created at the service's startup. Another approach is to run a series of commands in a separate Docker Compose service, which is definitely more complicated. The network_mode is set to bridge to allow the containers spawned by the runner to access the MinIO instance. More information about this can be found at Docker executor is unable to connect to a local MinIO container.

Finally, it's time for the GitLab runner service to be defined:

version: "3.9"

services:
  minio:
    build:
      context: .
      dockerfile: minio.Dockerfile
    container_name: runner-cache
    ports:
      - "9000:9000"
      - "9001:9001"
    volumes:
      - minio_data:/data
    environment:
      MINIO_ROOT_USER: minio
      MINIO_ROOT_PASSWORD: minio123
    command: server --console-address ":9001" /data
    network_mode: bridge

  runner:
    build: 
      context: .
      dockerfile: runner.Dockerfile
    image: runner
    container_name: runner
    environment:
      - GITLAB_TOKEN=AA0000000AaaaAaaaaAAAaAAA0Aaa
      - CONCURRENT_JOBS=2
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - runner_config:/etc/gitlab-runner
    restart: always
    depends_on:
      - minio

  portainer:
    image: portainer/portainer-ce:latest
    container_name: portainer
    ports:
      - "8000:8000"
      - "9443:9443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - portainer_data:/data
    restart: always

volumes:
  portainer_data:
  runner_config:
  minio_data:

docker-compose.yml

FROM gitlab/gitlab-runner:alpine
COPY ./config.template.toml /tmp/config.template.toml
COPY --chmod=0777 entrypoint /entrypoint

runner.Dockerfile

#!/bin/bash

# gitlab-runner data directory
DATA_DIR="/etc/gitlab-runner"
CONFIG_FILE=${CONFIG_FILE:-$DATA_DIR/config.toml}
# custom certificate authority path
CA_CERTIFICATES_PATH=${CA_CERTIFICATES_PATH:-$DATA_DIR/certs/ca.crt}
LOCAL_CA_PATH="/usr/local/share/ca-certificates/ca.crt"

update_ca() {
  echo "Updating CA certificates..."
  cp "${CA_CERTIFICATES_PATH}" "${LOCAL_CA_PATH}"
  update-ca-certificates --fresh >/dev/null
}

if [ -f "${CA_CERTIFICATES_PATH}" ]; then
  # update the ca if the custom ca is different than the current
  cmp --silent "${CA_CERTIFICATES_PATH}" "${LOCAL_CA_PATH}" || update_ca
fi

# register runner
gitlab-runner register \
  --template-config /tmp/config.template.toml \
  --non-interactive \
  --executor "docker" \
  --docker-image mcr.microsoft.com/dotnet/sdk:5.0 \
  --url "https://gitlab.com/" \
  --registration-token "${GITLAB_TOKEN}" \
  --run-untagged="true" \
  --locked="false" \
  --access-level="not_protected"

sed -i "s/concurrent.*/concurrent = ${CONCURRENT_JOBS}/" ${DATA_DIR}/config.toml

# start runner
gitlab-runner run --user=gitlab-runner --working-directory=/home/gitlab-runner

# unregister runner
gitlab-runner unregister --all-runners

entrypoint

[[runners]]
  [runners.docker]
    links = ["runner-cache:minio"]
  [runners.cache]
    Type = "s3"
    Shared = true
    [runners.cache.s3]
      AccessKey = "minio"
      SecretKey = "minio123"
      BucketName = "runner"
      Insecure = true
      ServerAddress = "minio:9000"

config.template.toml

The custom Dockerfile for the runner allows us to do three things:

pass the caching service configuration using config.template.toml file
set the concurrency of the runner as by default it would only run 1 job at once (that's the sed command and unfortunately there is no better way of doing it)
register the runner with GitLab and unregister it during a graceful shutdown

The entrypoint script is a slightly modified version of a script that's used in the official gitlab/gitlab-runner image.

And that's it. Now just run:

docker compose build --no-cache
docker compose up -d

To verify that the runner was registered successfully, go to the Settings tab and CI/CD menu:

If it's not there, open Portainer which should be running at 127.0.0.1:9443 and inspect the runner container logs.

To shut everything down, run:

docker compose down

Verify whether the runner was unregistered by going to the Runners section in the CI/CD menu once more:

The custom runners are about 3 to 4 times faster for me than GitLab's shared runners, so it is definitely recommended to use them! Here are all the files we created together for easier viewing:

GitLab runner image

GitLab runner image. GitHub Gist: instantly share code, notes, and snippets.

Gist262588213843476

Cover photo by Rinke Dohmen on Unsplash