Overview
This article provides local configuration details specific to the site. Links to the relevant documentation will also be provided.
Description
The gitlab-runner is a tool that uses the .gitlab-ci.yml file to build, test, and deploy to the target host. Each job is unique to itself but if it fails, all subsequent jobs do not run. A gitlab-runner is similar to a Jenkins Agent. You want to install it on a server different than the main gitlab server so workloads don’t impact access to gitlab itself.
Runner Installation
Installation is pretty easy. After you create the new runner server, you pull and install the runner for the server then register it.
Pull the runner binary
# curl -LJO "https://gitlab-runner-downloads.s3.amazonaws.com/latest/rpm/gitlab-runner_amd64.rpm"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 418M 100 418M 0 0 1583k 0 0:04:30 0:04:30 --:--:-- 1807kInstall the runner.
# rpm -ivh gitlab-runner_amd64.rpm
warning: gitlab-runner_amd64.rpm: Header V4 RSA/SHA512 Signature, key ID 35dfa027: NOKEY
Preparing... ################################# [100%]
Updating / installing...
1:gitlab-runner-14.6.0-1 ################################# [100%]
GitLab Runner: creating gitlab-runner...
Home directory skeleton not used
Runtime platform arch=amd64 os=linux pid=11354 revision=5316d4ac version=14.6.0
gitlab-runner: the service is not installed
Runtime platform arch=amd64 os=linux pid=11363 revision=5316d4ac version=14.6.0
gitlab-ci-multi-runner: the service is not installed
Runtime platform arch=amd64 os=linux pid=11387 revision=5316d4ac version=14.6.0
Runtime platform arch=amd64 os=linux pid=11423 revision=5316d4ac version=14.6.0
INFO: Docker installation not found, skipping clear-docker-cacheThen register the runner (this is internal to my homelab so the token being displayed isn’t an issue).
# gitlab-runner register
Runtime platform arch=amd64 os=linux pid=11468 revision=5316d4ac version=14.6.0
Running in system-mode.
Enter the GitLab instance URL (for example, https://gitlab.com/):
http://lnmt1cuomgitlab.internal.pri/
Enter the registration token:
Li7r2znM5yVedatwy7Uy
Enter a description for the runner:
[lnmt1cuomglrunr1.internal.pri]:
Enter tags for the runner (comma-separated):
local
Registering runner... succeeded runner=Li7r2znM
Enter an executor: docker, docker-ssh, virtualbox, docker+machine, docker-ssh+machine, kubernetes, custom, parallels, shell, ssh:
shell
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!Final Configuration
Once it’s registered, you’ll need to create a rsa key pair and copy it to whatever target servers you intend to deploy jobs to. In my example, I have a local server where I can test to make sure things work, and the remote live site. Log in to the two servers to register the host key. I’m using php in this case so the server also needs to have php installed in order to do my minimal lint test of the php scripts.
Note to also get the rsa public keys for any of the artifact servers you pull from. For example I have artifacts on my two dev servers. The means all the gitlab-runner servers that pull from those two dev servers will need their rsa public keys added to the dev servers.
Create Jobs
You’ll need to create a .gitlab-ci.yml file in your repository that contains the steps required to build the project. In this case, I’m using my small Llamas band website for the example but it could be anything.
Here I define the stages I’ll be using for the deployment.
Each job is unique to the other jobs. Any task you want to do such as removing the .git directory, will need to be done in each stage. Using tags, you could point each stage to different runners.
stages:
- test
- deploy-local
- deploy-remote
test-job:
tags:
- test
stage: test
script:
- |
for i in $(find "${CI_PROJECT_DIR}" -type f -name \*.php -print)
do
php -l ${i}
done
deploy-local-job:
tags:
- home
stage: deploy-local
script:
- rm -rf "${CI_PROJECT_DIR}"/.git
- rm -f "${CI_PROJECT_DIR}"/.gitlab-ci.yml
- /usr/bin/rsync -av --delete --no-perms --no-owner --no-group --omit-dir-times --rsync-path=/usr/bin/rsync "${CI_PROJECT_DIR}"/ svcacct@ndld1cuomtool11:/var/www/html/llamas/
deploy-remote-job:
tags:
- remote
stage: deploy-remote
script:
- rm -rf "${CI_PROJECT_DIR}"/.git
- rm -f "${CI_PROJECT_DIR}"/.gitlab-ci.yml
- /usr/bin/rsync -av --delete --no-perms --no-owner --no-group --omit-dir-times --rsync-path=/usr/bin/rsync "${CI_PROJECT_DIR}"/ svcacct@remote:/usr/local/httpd/llamas/Pipelines
When you check in the .gitlab-ci.yml file, a pipeline starts. In the project, on the left side, click on the CI/CD and then Pipelines to see the pipeline progress. Note that there is a CI Lint button where you can validate your .gitlab-ci.yml file.
You can see the failed pipeline, due to incorrect spacing for the test script (I verified in the CI Lint section). After fixing, the pipeline passed.
Clicking on the Passed or Failed button will take you to the Pipeline.
You can see the progress of the pipeline. Each stage can be rerun by clicking on the arrow-circle and you can see how the task worked by clicking on the stage.
This is the test-job stage. Line 2 shows it’s running on the dedicated runner server. It’s a ‘Shell’ executor. It pulls the git repo to the working directory. Then runs the quick php lint test on the three files.
Things to Think About
With each stage being a unique task, we could have a runner that only does testing. It would have all the necessary tools to test projects such as php in this case. You could also have a dedicated runner that has access to the local QA box but no access to any other server. Same with remote access. You create tags for test-runner, local-qa, and remote-live for example. Then the three stages in the above example would have appropriate functions.
References
- https://docs.gitlab.com/runner/
- https://docs.gitlab.com/runner/install/linux-manually.html
- https://docs.gitlab.com/runner/register/index.html#linux
- https://docs.gitlab.com/ee/ci/yaml/gitlab_ci_yaml.html