Installation | Topics | Beyond Lino

Hosting your own GitLab runner

How to set up a self-managed GitLab group runner

  1. Install docker docker.io:

    $ sudo apt install docker docker.io

  2. Add gitlab runner repository in the package manager’s repositories:

    $ # You can probably skip this if you are on ubuntu, check
$ # if this `apt list gitlab-runner` command fetches any result
$ curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | sudo bash

  3. Install gitlab runner:

    $ sudo apt install gitlab-runner

(4) The options a) and b) below are mutually exclusive. Do only one of them but never both.

(a) Do this section if you are installing gitlab-runner on a personal computer:

  • Get openvpn configuration file to connect your computer with our private network from Sharif.

  • Install openvpn on your computer:

    $ sudo apt-get install openvpn

  • Put the openvpn cofiguration file into the /etc/openvpn directory.

    For example, if your configuration filename is myvpn.conf and it is corrently in $HOME/Downloads directory, then, do:

    $ sudo mv ~/Downloads/myvpn.conf /etc/openvpn/

  • Enable the configuration file to run as a systemd service:

    $ sudo systemctl enable openvpn@myvpn.service

  • Start the openvpn service:

    $ sudo service openvpn@myvpn start

  • Check if the service is running:

    $ ip addr show dev tun0

  • Edit the /etc/default/openvpn file and put myvpn as a value of the AUTOSTART option.

    This is required to run the myvpn service on restart automatically.

  • Create the following directory:

    $ sudo mkdir /etc/systemd/system/gitlab-runner.service.d

  • Put the proxy environment variables as given below:

    $ printf '[Service]\nEnvironment="HTTP_PROXY=http://10.8.0.1:3131/"\nEnvironment="HTTPS_PROXY=http://10.8.0.1:3131/"\n' | sudo tee /etc/systemd/system/gitlab-runner.service.d/http-proxy.conf &> /dev/null

  • Reload systemd daemon:

    $ sudo systemctl daemon-reload

  • Restart gitlab-runner service:

    $ sudo service gitlab-runner restart

  • Register gitlab runner:

    $ sudo HTTP_PROXY=http://10.8.0.1:3131/ HTTPS_PROXY=http://10.8.0.1:3131/ gitlab-runner register

P.S. The above example assumes that the user does not have any other vpn service running. When the assumption is not true, some example commands may vary to some degree. In which case, the user should run the commands as deems appropriate.

  1. Register gitlab runner on a computer with a public IP address:

    $ sudo gitlab-runner register

The last command from either option a) or b) will prompt for some inputs. Give the values described below to the associated inputs:

> url: https://gitlab.com/
> registration-token: (the token from lino-framework.org:/var/gitlab_runner_group_registration_token)
> executor: docker
> docker-image: debian:latest
> description: (give any description to your liking)
> tag-list: lino,linorunner

To get the registration-token from lino-framework.org, use:

$ ssh username@lino-framework.org
$ sudo cat /var/gitlab_runner_group_registration_token

Leave (just press ENTER/RETURN on) the other input prompts.

(5) Create a file /etc/cron.daily/prune_docker.sh and add the following content:

#!/bin/sh
docker system prune -af
docker volume prune -af

Make the file executable:

sudo chmod a+x /etc/cron.daily/prune_docker.sh

Cheat sheet

See also Using Docker to test getlino.

Check the status of services

service gitlab-runner status and service docker status might be useful.

Example output:

# service gitlab-runner status
● gitlab-runner.service - GitLab Runner
     Loaded: loaded (/etc/systemd/system/gitlab-runner.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2023-12-07 06:02:16 CET; 8min ago
   Main PID: 2794794 (gitlab-runner)
      Tasks: 7 (limit: 2300)
     Memory: 35.9M
        CPU: 1.415s
     CGroup: /system.slice/gitlab-runner.service
             └─2794794 /usr/bin/gitlab-runner run --working-directory /home/gitlab-runner --config /etc/gitlab-runner/config.toml --service gitlab-runner --user gitlab-runner

dets  07 06:09:45 kuu gitlab-runner[2794794]: Job succeeded                                       duration_s=93.188262843 job=5691656362 project=25842361 runner=eyNAxzsXP
dets  07 06:09:45 kuu gitlab-runner[2794794]: Appending trace to coordinator...ok                 code=202 job=5691656362 job-log=0-31119 job-status=running runner=eyNAxzsXP sent-log=22035-31118 status=2>
dets  07 06:09:45 kuu gitlab-runner[2794794]: Updating job...                                     bytesize=31119 checksum=crc32:8a1f0b3d job=5691656362 runner=eyNAxzsXP
dets  07 06:09:45 kuu gitlab-runner[2794794]: Submitting job to coordinator...accepted, but not yet completed  bytesize=31119 checksum=crc32:8a1f0b3d code=202 job=5691656362 job-status= runner=eyNAxzsXP >
dets  07 06:09:46 kuu gitlab-runner[2794794]: Updating job...                                     bytesize=31119 checksum=crc32:8a1f0b3d job=5691656362 runner=eyNAxzsXP
dets  07 06:09:47 kuu gitlab-runner[2794794]: Submitting job to coordinator...ok                  bytesize=31119 checksum=crc32:8a1f0b3d code=200 job=5691656362 job-status= runner=eyNAxzsXP update-interv>
dets  07 06:09:47 kuu gitlab-runner[2794794]: Removed job from processing list                    builds=0 job=5691656362 max_builds=1 project=25842361 repo_url=https://gitlab.com/lino-framework/avanti.g>
dets  07 06:09:47 kuu gitlab-runner[2794794]: Checking for jobs... received                       job=5691656364 repo_url=https://gitlab.com/lino-framework/avanti.git runner=eyNAxzsXP
dets  07 06:09:47 kuu gitlab-runner[2794794]: Added job to processing list                        builds=1 job=5691656364 max_builds=1 project=25842361 repo_url=https://gitlab.com/lino-framework/avanti.g>
dets  07 06:09:51 kuu gitlab-runner[2794794]: Appending trace to coordinator...ok                 code=202 job=5691656364 job-log=0-266 job-status=running runner=eyNAxzsXP sent-log=0-265 status=202 Accep>


# service docker status
● docker.service - Docker Application Container Engine
     Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2023-10-17 20:11:26 CEST; 1 months 20 days ago
TriggeredBy: ● docker.socket
       Docs: https://docs.docker.com
   Main PID: 222529 (dockerd)
      Tasks: 11
     Memory: 145.0M
        CPU: 30min 10.468s
     CGroup: /system.slice/docker.service
             └─222529 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

dets  07 06:09:53 kuu dockerd[222529]: time="2023-12-07T06:09:53.777139918+01:00" level=info msg="ignoring event" container=d44dcea157375ab03dc565ad4eda6f26c03cf6f7f1fe85093bde9a739b1110b1 module=libcont>
dets  07 06:09:55 kuu dockerd[222529]: time="2023-12-07T06:09:55.767585445+01:00" level=info msg="ignoring event" container=d44dcea157375ab03dc565ad4eda6f26c03cf6f7f1fe85093bde9a739b1110b1 module=libcont>
dets  07 06:11:15 kuu dockerd[222529]: time="2023-12-07T06:11:15.477822794+01:00" level=info msg="ignoring event" container=d15fab7a3fae2a7aa11c4d5dbae3177709b0c70654b4280d77dfe3c5c551c5d4 module=libcont>
dets  07 06:11:18 kuu dockerd[222529]: time="2023-12-07T06:11:18.895624674+01:00" level=info msg="ignoring event" container=d44dcea157375ab03dc565ad4eda6f26c03cf6f7f1fe85093bde9a739b1110b1 module=libcont>
dets  07 06:11:19 kuu dockerd[222529]: time="2023-12-07T06:11:19.426309728+01:00" level=info msg="ignoring event" container=d44dcea157375ab03dc565ad4eda6f26c03cf6f7f1fe85093bde9a739b1110b1 module=libcont>
dets  07 06:11:24 kuu dockerd[222529]: time="2023-12-07T06:11:24.809602860+01:00" level=info msg="ignoring event" container=137c32e1964e31ed768b6621d408a010a02c8ba16e5c3646e0b1d8361e43b398 module=libcont>
dets  07 06:11:25 kuu dockerd[222529]: time="2023-12-07T06:11:25.550667054+01:00" level=info msg="ignoring event" container=fd88cc7e3aec24317e3719c364b7b2a8e8aca803af75e184e8037d7a636654fd module=libcont>
dets  07 06:11:27 kuu dockerd[222529]: time="2023-12-07T06:11:27.280672302+01:00" level=info msg="ignoring event" container=ccac0c97cc937801d8204b15db7dc7b1f201d7b65e665281867b9760b5ea6d60 module=libcont>
dets  07 06:11:28 kuu dockerd[222529]: time="2023-12-07T06:11:28.702317232+01:00" level=info msg="ignoring event" container=ccac0c97cc937801d8204b15db7dc7b1f201d7b65e665281867b9760b5ea6d60 module=libcont>
dets  07 06:12:45 kuu dockerd[222529]: time="2023-12-07T06:12:45.727563994+01:00" level=info msg="ignoring event" container=42816e438eadfaa40b9340995f770504591d674c1c17c1e22d151132722a3105 module=libcont>

Docker stats

docker stats displays a live stream of container(s) resource usage statistics, in a way that reminds top.

You can say --no-stream to disable streaming stats and only pull the first result.

Example:

# docker stats --no-stream
CONTAINER ID   NAME                                                                                       CPU %     MEM USAGE / LIMIT     MEM %     NET I/O           BLOCK I/O       PIDS
b8642fb7b8eb   runner-eynaxzsxp-project-26156760-concurrent-0-07c157edbea57fc8-mysql-0-wait-for-service   0.05%     10.43MiB / 1.892GiB   0.54%     3.01kB / 3.19kB   2.66MB / 0B     4
57de781f8b48   runner-eynaxzsxp-project-26156760-concurrent-0-07c157edbea57fc8-mysql-0                    51.96%    436.2MiB / 1.892GiB   22.51%    3.92kB / 2.24kB   103MB / 233MB   34

List the configured runners

Use gitlab-runner list to list all configured runners.

Example:

# gitlab-runner list
Runtime platform                                    arch=amd64 os=linux pid=2794711 revision=d89a789a version=16.4.1
Listing configured runners                          ConfigFile=/etc/gitlab-runner/config.toml
kuu                                                 Executor=docker Token=glrt-eyNAxzsXPzK3wpC2WN2j URL=https://gitlab.com