I was looking for a solution to run a GitLab pipeline locally. I haven’t really figured out an easy way, but apparently one could use the gitlab-runner tool to run individual jobs. Although you can install all tools for Windows I wanted to run the tools a bit more isolated. Therefore I decided to use wsl.
This is what I had to do!
- install Ubuntu distribution
- install gitlab-runner tools
- install docker
- run the gitlab commands
The list is quite short, but I spent quite some time figuring out how I can make caching happen.
In a nutshell I run an Ubuntu VM using wsl in which I can execute my pipeline jobs using gitlab-runner. The runner is spinning up Docker containers to execute the jobs as declared in .gitlab-ci.yml.
Ubuntu / WSL
First I had to install the Ubuntu WSL distro. Although the command line tells me where to find the distros (i.e. the Microsoft Store) I had a bit a hard time finding it. But the link WSL | Ubuntu helped me out as there is a link to directly get to the proper distro.
I have a complete Ubuntu environment ready in seconds and the integration with Windows works really well. I start WSL by typing wsl -d Ubuntu in my command line.
Install the tools
First of all I installed gitlab-runner:
sudo apt install gitlab-runner
Then I installed docker, which is a bit of a pain if you just want to get started quickly. I basically followed this guide and it worked well: How To Install and Use Docker on Ubuntu 20.04 | DigitalOcean
WSL 2
I first tried to run docker on my VM, but it failed. I had to upgrade my distro to WSL 2 by invoking this command:
wsl –set-version Ubuntu 2
After launching the VM again, I was able to run docker commands.
Docker
When I run my GitLab pipeline, I want to use Docker as executor. GitLab runner basically spins up containers (as per image defined in the .gitlab-ci.yml) and executes the job. The Docker daemon doesn’t start automatically, this is not hard to configure, but to first test my setup I had to start it manually by invoking sudo service docker start
I verified my setup by running docker run hello-world. If it works, it will print something like:
Running GitLab
Although it reads pretty simple, I spent quite some time understanding how to use the gitlab-runner tool. My main issue was to ensure the cache is working between the job executions. All the builds runs in a container and my initial assumption that caching just works was wrong. The tool tells me that instead of a distributed cache a local cache is used, but it never worked.
The trick is to mount a volume, so that the cache created inside the container is persisted on the host.
So, to actually run a job from my pipeline I navigated to a project with the .gitlab-ci.yml in it and executed the following command:
sudo gitlab-runner exec docker build-web –docker-volumes /home/gitlab-runner/cache/:/cache
Where build-web is the job I want to run and /home/gitlab-runner/cache the directory on the host system where the cache should be stored. By default the runner will put the cache in the /cache directory in the container.
Final Thoughts
I was hoping that I can execute the whole pipeline using the command line. Seems with gitlab-runner I can only run a single job. Still good to test stuff – definitely good to learn more about how GitLab runners work. And maybe this guide helps someone setting up their local GitLab runner.