Skip to content

GitLab CI template for Docker

This project implements a generic GitLab CI template Docker based projects.

Usage

In order to include this template in your project, add the following to your .gitlab-ci.yml :

include:
  - project: 'to-be-continuous/docker'
    ref: '2.2.0'
    file: '/templates/gitlab-ci-docker.yml'

Understanding the Docker template

The template supports two ways of building your Docker images:

  1. The former Docker-in-Docker technique, that was widely used for years because of no other alternative, but that is now commonly recognized to have significant security issues (read this post for more info),
  2. Or using kaniko, an open-source tool from Google for building Docker images, and that solves Docker-in-Docker security issues (and also speeds-up build times).

By default, the template uses the kaniko way, but you may activate the Docker-in-Docker build at your own risks by setting DOCKER_DIND_BUILD to true (see below). ⚠ī¸ In that case, make sure your runner has required privileges to run Docker-in-Docker (see GitLab doc).

Global variables

The Docker template uses some global configuration used throughout all jobs.

Name description default value
DOCKER_DIND_BUILD Set to enable Docker-in-Docker build (⚠ī¸ unsecured, requires privileged runners). (none) (kaniko build by default)
DOCKER_KANIKO_IMAGE The Docker image used to run kaniko - for kaniko build only gcr.io/kaniko-project/executor:debug (use debug images for GitLab)
DOCKER_IMAGE The Docker image used to run the docker client (see full list) - for Docker-in-Docker build only docker:latest
DOCKER_DIND_IMAGE The Docker image used to run the Docker daemon (see full list) - for Docker-in-Docker build only docker:dind
DOCKER_FILE The path to your Dockerfile ./Dockerfile
DOCKER_CONTEXT_PATH The Docker context path (working directory) none only set if you want a context path different from the Dockerfile location

In addition to this, the template supports standard Linux proxy variables:

Name description default value
http_proxy Proxy used for http requests none
https_proxy Proxy used for https requests none
no_proxy List of comma-separated hosts/host suffixes none

Images

For each Dockerfile, the template builds an image that may be pushed as two distinct images, depending on a certain workflow:

  1. snapshot: the image is first built from the Dockerfile and then pushed to some Docker registry as the snapshot image. It can be seen as the raw result of the build, but still untested and unreliable.
  2. release: once the snapshot image has been thoroughly tested (both by package-test stage jobs and/or acceptance stage jobs after being deployed to some server), then the image is pushed one more time as the release image. This second push can be seen as the promotion of the snapshot image being now tested and reliable.

In practice:

  • the snapshot image is always pushed by the template (pipeline triggered by a Git tag or commit on any branch),
  • the release image is only pushed:
    • on a pipeline triggered by a Git tag,
    • on a pipeline triggered by a Git commit on master.

The snapshot and release images are defined by the following variables:

Name description default value
DOCKER_SNAPSHOT_IMAGE Docker snapshot image $CI_REGISTRY_IMAGE/snapshot:$CI_COMMIT_REF_SLUG
DOCKER_RELEASE_IMAGE Docker release image $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME

As you can see, the Docker template is configured by default to use the GitLab container registry. You may perfectly override this and use another Docker registry, but be aware of a few things:

  • the DOCKER_SNAPSHOT_IMAGE requires a Docker registry that allows tag overwrite,
  • the DOCKER_RELEASE_IMAGE may use a Docker registry that doesn't allow tag overwrite, but:
    1. you should avoid overwriting a Git tag (at it will obviously fail while trying to (re)push the Docker image),
    2. you have to deactivate publish on master branch by setting the $PUBLISH_ON_PROD variable to false (as it would lead to the master tag being overwritten).

Registries and credentials

As seen in the previous chapter, the Docker template uses by default the GitLab registry to push snapshot and release images. Thus it makes use of credentials provided by GitLab itself to login (CI_REGISTRY_USER / CI_REGISTRY_PASSWORD).

But when using other registry(ies), you'll have also to configure appropriate Docker credentials.

Using the same registry for snapshot and release

If you use the same registry for both snapshot and release images, you shall use the following configuration variables:

Name description
🔒 DOCKER_REGISTRY_USER Docker registry username for image registry
🔒 DOCKER_REGISTRY_PASSWORD Docker registry password for image registry

Using different registries for snapshot and release

If you use different registries for snapshot and release images, you shall use separate configuration variables:

Name description
🔒 DOCKER_REGISTRY_SNAPSHOT_USER Docker registry username for snapshot image registry
🔒 DOCKER_REGISTRY_SNAPSHOT_PASSWORD Docker registry password for snapshot image registry
🔒 DOCKER_REGISTRY_RELEASE_USER Docker registry username for release image registry
🔒 DOCKER_REGISTRY_RELEASE_PASSWORD Docker registry password for release image registry

Multi Dockerfile support

This template supports building multiple Docker images from a single Git repository.

You can define the images to build using the parallel matrix jobs pattern inside the .docker-base job (this is the top parent job of all Docker template jobs).

Since each job in the template extends this base job, the pipeline will produce one job instance per image to build. You can independently configure each instance of these jobs by redefining the variables described throughout this documentation.

For example, if you want to build two Docker images, you must specify where the Dockerfiles are located and where the resulting images will be stored. You can do so by adding a patch to the .docker-base job in your .gitlab-ci.yml file so that it looks like this:

.docker-base:
  parallel:
    matrix:
    - DOCKER_FILE: "front/Dockerfile"
      DOCKER_SNAPSHOT_IMAGE: "$CI_REGISTRY/$CI_PROJECT_PATH/front:$CI_COMMIT_REF_SLUG"
      DOCKER_RELEASE_IMAGE: "$CI_REGISTRY/$CI_PROJECT_PATH/front:$CI_COMMIT_REF_NAME"
    - DOCKER_FILE: "back/Dockerfile"
      DOCKER_SNAPSHOT_IMAGE: "$CI_REGISTRY/$CI_PROJECT_PATH/back:$CI_COMMIT_REF_SLUG"
      DOCKER_RELEASE_IMAGE: "$CI_REGISTRY/$CI_PROJECT_PATH/back:$CI_COMMIT_REF_NAME"

If you need to redefine a variable with the same value for all your Dockerfiles, you can just declare this variable as a global variable. For example, if you want to build all your images using Docker-in-Docker, you can simply define the DOCKER_DIND_BUILD variable as a global variable:

variables:
  DOCKER_DIND_BUILD: "True"

Secrets management

Here are some advices about your secrets (variables marked with a 🔒):

  1. Manage them as project or group CI/CD variables:
    • masked to prevent them from being inadvertently displayed in your job logs,
    • protected if you want to secure some secrets you don't want everyone in the project to have access to (for instance production secrets).
  2. In case a secret contains characters that prevent it from being masked, simply define its value as the Base64 encoded value prefixed with @b64@: it will then be possible to mask it and the template will automatically decode it prior to using it.
  3. Don't forget to escape special characters (ex: $ -> $$).

Jobs

docker-lint job

This job performs a Lint on your Dockerfile.

It is bound to the build stage, and uses the following variables:

Name description default value
DOCKER_LINT_IMAGE The dockerlint image projectatomic/dockerfile-lint:latest
DOCKER_LINT_ARGS Additional dockerfile_lint arguments (none)

In case you have to disable some rules, copy and edit the rules into mycustomdockerlint.yml and set DOCKER_LINT_ARGS: '-r mycustomdockerlint.yml'

docker-hadolint job

This job performs a Lint on your Dockerfile.

It is bound to the build stage, and uses the following variables:

Name description default value
DOCKER_HADOLINT_IMAGE The Hadolint image hadolint/hadolint:latest-alpine
DOCKER_HADOLINT_ARGS Additional hadolint arguments ``

In case you have to disable some rules, either add --ignore XXXX to the DOCKER_HADOLINT_ARGS variable or create a Hadolint configuration file named hadolint.yaml at the root of your repository.

You can also use inline ignores in your Dockerfile:

# hadolint ignore=DL3006
FROM ubuntu

# hadolint ignore=DL3003,SC1035
RUN cd /tmp && echo "hello!"

docker-build job

This job builds the image and publishes it to the snapshot repository.

It is bound to the package-build stage, and uses the following variables:

Name description default value
DOCKER_BUILD_ARGS Additional docker build/kaniko arguments (none)
DOCKER_REGISTRY_MIRROR URL of a Docker registry mirror to use during the image build (instead of default https://index.docker.io) (none)
DOCKER_METADATA Additional docker build/kaniko arguments to set label OCI Image Format Specification

This job produces an output variable that is propagated to downstream jobs (using dotenv artifacts):

  • docker_image: set to $DOCKER_SNAPSHOT_IMAGE

It may be freely used in downstream jobs (for instance to deploy the upstream built Docker image, whatever the branch of tag).

If you want to use GitLab CI variables or any other variable in your Dockerfile, you can add them to DOCKER_BUILD_ARGS like so:

DOCKER_BUILD_ARGS: "--build-arg CI_PROJECT_URL --build-arg MY_VAR='MY_VALUE'"

These variables will then be available for use in your Dockerfile:

FROM scratch

ARG CI_PROJECT_URL
ARG MY_VAR
LABEL name="my-project"                   \
      description="My Project: $MY_VAR"   \
      url=$CI_PROJECT_URL                 \
      maintainer="my-project@acme.com"

Default value for DOCKER_METADATA supports a subset of the OCI Image Format Specification for labels and use GitLab CI pre-defined variables to guess the value as follow :

Label Gitlab CI pre-defined variable
org.opencontainers.image.url $CI_PROJECT_URL
org.opencontainers.image.source $CI_PROJECT_URL
org.opencontainers.image.title $CI_PROJECT_PATH
org.opencontainers.image.ref.name $CI_COMMIT_REF_NAME
org.opencontainers.image.revision $CI_COMMIT_SHA
org.opencontainers.image.created $CI_JOB_STARTED_AT

Note that spaces are currently not supported by Kaniko. Therefore, title couldn't be CI_PROJECT_TITLE.

You may disable this feature by setting DOCKER_METADATA to empty or you can override some of the pre-defined label value with the DOCKER_BUILD_ARGS.

DOCKER_BUILD_ARGS: "--label org.opencontainers.image.title=my-project"

If you have defined one of those labels in the Dockerfile, the final value will depend if image is built with Kaniko or Docker in Docker. With Kaniko, the value of the Dockerfile take precedence, while with DinD command-line argument take precedence.

docker-healthcheck job

⚠ī¸ this job requires that your runner has required privileges to run Docker-in-Docker. If it is not the case this job will not be run.

This job performs a Health Check on your built image.

It is bound to the package-test stage, and uses the following variables:

Name description default value
DOCKER_HEALTHCHECK_DISABLED Set to true to disable health check (none: enabled by default)
DOCKER_HEALTHCHECK_TIMEOUT When testing a Docker Health (test stage), how long (in seconds) wait for the HealthCheck status 60
DOCKER_HEALTHCHECK_OPTIONS Docker options for health check such as port mapping, environment... (none)
DOCKER_HEALTHCHECK_CONTAINER_ARGS Set arguments sent to the running container for health check (none)

In case your Docker image is not intended to run as a service and only contains a client tool (like curl, Ansible, ...) you can test it by overriding the Health Check Job. See this example.

⚠ī¸ Keep in mind that the downloading of the snapshot image by the GitLab runner will be done during the waiting time (max DOCKER_HEALTHCHECK_TIMEOUT). In case your image takes quite some time to be downloaded by the runner, increase the value of DOCKER_HEALTHCHECK_TIMEOUT in your .gitlab-ci.yml file.

docker-trivy job

This job performs a Vulnerability Static Analysis with Trivy on your built image.

⚠ī¸ As presented below, this job is enabled only if you specify a Trivy server address with the DOCKER_TRIVY_ADDR environment variable.

A Trivy server has been deployed internally. If you want to use it, you can add the following variable definition to you .gitlab-ci.yml:

variables:
  DOCKER_TRIVY_ADDR: "https://trivy.acme.host"

It is bound to the package-test stage, and uses the following variables:

Name description default value
DOCKER_TRIVY_IMAGE The docker image used to scan images with Trivy aquasec/trivy:latest
DOCKER_TRIVY_ADDR The Trivy server address (none: disabled by default)
DOCKER_TRIVY_SECURITY_LEVEL_THRESHOLD Severities of vulnerabilities to be displayed (comma separated values: UNKNOWN, LOW, MEDIUM, HIGH, CRITICAL) UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL
DOCKER_TRIVY_DISABLED Set to true to disable Trivy analysis (none)
DOCKER_TRIVY_ARGS Additional trivy client arguments --ignore-unfixed

docker-publish job

This job pushes (promotes) the built image as the release image skopeo.

Name description default value
DOCKER_SKOPEO_IMAGE The Docker image used to run skopeo quay.io/skopeo/stable:latest
DOCKER_PUBLISH_ARGS Additional skopeo copy arguments (none)
AUTODEPLOY_TO_PROD Set to enable automatic publish (and deploy) on master branch none (enabled)
PUBLISH_ON_PROD Determines whether this job is enabled on master branch true_ (enabled)

This job produces an output variable that is propagated to downstream jobs (using dotenv artifacts):

  • docker_image: set to $DOCKER_RELEASE_IMAGE

It may be freely used in downstream jobs (for instance to deploy the upstream built Docker image, whatever the branch of tag).

Examples

Using the GitLab Docker registry

This sample is the easiest one as you just have nothing to do.

All template variables are configured by default to build and push your Docker images on the GitLab registry.

Using an external Docker registry

With this template, you may perfectly use an external Docker registry (ex: a JFrog Artifactory, a private Kubernetes registry, ...).

Here is a .gitlab-ci.yaml using an external Docker registry:

include:
  - project: 'to-be-continuous/docker'
    ref: '2.2.0'
    file: '/templates/gitlab-ci-docker.yml'

variables:
  DOCKER_SNAPSHOT_IMAGE: "registry.acme.host/$CI_PROJECT_NAME/snapshot:$CI_COMMIT_REF_SLUG"
  DOCKER_RELEASE_IMAGE: "registry.acme.host/$CI_PROJECT_NAME:$CI_COMMIT_REF_NAME"
  # $DOCKER_REGISTRY_USER and $DOCKER_REGISTRY_PASSWORD are defined as secret GitLab variables

Depending on the Docker registry you're using, you may have to use a real password or generate a token as authentication credential.

Building multiple Docker images

Here is a .gitlab-ci.yaml that builds 2 Docker images from the same project (uses parallel matrix jobs):

include:
  - project: 'to-be-continuous/docker'
    ref: '2.2.0'
    file: '/templates/gitlab-ci-docker.yml'

variables:
  DOCKER_DIND_BUILD: "True"

.docker-base:
  parallel:
    matrix:
    - DOCKER_FILE: "front/Dockerfile"
      DOCKER_SNAPSHOT_IMAGE: "$CI_REGISTRY/$CI_PROJECT_PATH/front/snapshot:$CI_COMMIT_REF_SLUG"
      DOCKER_RELEASE_IMAGE: "$CI_REGISTRY/$CI_PROJECT_PATH/front:$CI_COMMIT_REF_NAME"
    - DOCKER_FILE: "back/Dockerfile"
      DOCKER_SNAPSHOT_IMAGE: "$CI_REGISTRY/$CI_PROJECT_PATH/back/snapshot:$CI_COMMIT_REF_SLUG"
      DOCKER_RELEASE_IMAGE: "$CI_REGISTRY/$CI_PROJECT_PATH/back:$CI_COMMIT_REF_NAME"

Gitlab compatibility

ℹī¸ This template is actually tested and validated on GitLab Community Edition instance version 13.12.11