This article aims to give you the knowledge needed to configure and test basic CI/CD pipelines using GitLab.
CI/CD pipelines are the top-level component of continuous integration, delivery, and deployment. Pipelines comprise of jobs and stages. A job defines what to do, for example, compile code. A stage determines what jobs are run, for example, a test stage could contain jobs to run unit tests for the frontend application and the backend application. Jobs that are within the same stage are executed in parallel.
A small Python app is used in this article to demonstrate pipelines however, the instructions can be adapted to the language of your choice. A link to the Python app is at the bottom of this article.
To follow along you will need a GitLab account and for the later section ‘testing locally’ you will need to install Docker and
Pipeline configurations are written in a file called
.gitlab-ci.yml using the Yaml syntax. This file must be located at the root of the application.
The pipeline configuration for our app is as follows. It’s a small app and the configuration is simple.
image: python:3.8 stages: - test test_app: stage: test script: - python -m pip install -r requirements.txt - pytest
The first line specifies the Docker image needed to run the jobs. We can use an image from Docker Hub by specifying the image-name and an optional tag. Above we have requested the Python image and used the 3.8 tag to make sure we are getting a version compatible with our app. If the tag is omitted then by default the latest version of the image will be used.
Next, the stages of the pipeline are defined. Our app contains one stage but a more sophisticated app may have more, such as build, test, deployment. Stages are used to group jobs, which are defined later.
The line beginning
test_app defines a job that is part of the
test stage. The
script section is where the commands for the job are defined. The
script section can take a single line or, as in the case above, an array of commands. Our job is made up of two commands first the dependencies listed in requirements.txt are installed using pip. Next, using Pytest the unit tests are run.
Testing the pipeline
At this point, it is worth making your first commit and pushing it to your repository on GitLab. GitLab will detect the presence of the
.gitlab-ci.yml file and runners will handle the pipeline.
You can view the progress of the pipeline by navigating to the pipelines section of the repo you’ve just pushed to. This section page can be found under CI/CD which is situated underneath ‘merge request’ on the sidebar in GitLab.
Testing the pipelines locally
Testing pipelines can be tedious if you’re having to make changes, make a commit and push to GitLab and wait for the results each time. Luckily, however, there is a better way to test jobs.
If you haven’t already, install Docker and
gitlab-runner. Once this is installed you can use it to run specific pipeline jobs locally. This greatly speeds up the feedback cycle.
sudo curl --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-darwin-amd64
Then grant the correct permissions with
sudo chmod +x /usr/local/bin/gitlab-runner
gitlab-runner is successfully installed, it can be used to run individual jobs.
gitlab-runner exec docker <job-name>
To run the
test_app job we defined earlier
gitlab-runner exec docker test_app
Conclusion and further reading
Hopefully, you found this article useful and you didn’t have much trouble getting your pipeline up and running. We defined a simple pipeline and used
gitlab-runner to test it locally.
Personally, I have found that using
gitlab-runner is much more convenient than pushing changes to GitLab each time. However, once I did find that I was getting different results locally compared to GitLab. I could get PHPStan to run locally, but it always failed on GitLab without providing any information about the issue.
If you have any questions or comments please feel free to reach out to me on Twitter @stuartmaynes.
GitLab’s documentation is comprehensive and worth reading if you want to continue your journey with pipelines and CI.
The code used is available on GitLab for you to use.