Skip to content

GitLab CI template for Source-to-Image

This project implements a GitLab CI/CD template to build reproducible container images from your source code with Source-to-Image (S2I).

Usage

This template can be used both as a CI/CD component or using the legacy include:project syntax.

Use as a CI/CD component

Add the following to your gitlab-ci.yml:

include:
  # 1: include the component
  - component: gitlab.com/to-be-continuous/s2i/gitlab-ci-s2i@1.1.1
    # 2: set/override component inputs
    inputs:
      builder-image: "fabric8/s2i-java:latest-java11" # ⚠ this is only an example

Use as a CI/CD template (legacy)

Add the following to your gitlab-ci.yml:

include:
  # 1: include the template
  - project: 'to-be-continuous/s2i'
    ref: '1.1.1'
    file: '/templates/gitlab-ci-s2i.yml'

variables:
  # 2: set/override template variables
  S2I_BUILDER_IMAGE: "fabric8/s2i-java:latest-java11" # ⚠ this is only an example

Understanding the S2I template

Global configuration

The S2I template uses some global configuration used throughout all jobs.

Input / Variable Description Default value
dind-image / S2I_DIND_IMAGE The Docker image used to run the Docker daemon registry.hub.docker.com/library/docker:dind
skopeo-image / S2I_SKOPEO_IMAGE The container image used to run skopeo quay.io/skopeo/stable:latest
version / S2I_VERSION The target S2I version to install/use latest
platform / S2I_PLATFORM The target S2I platform to install/use linux-386

Images

The S2I template builds a container image that may be pushed as two distinct images, depending on a certain workflow:

  1. snapshot: the image is first built and pushed to some container registry as the snapshot image. It can be seen as the raw result of the build, but still untested and unreliable.
  2. release: once the snapshot image has been thoroughly tested (both by package-test stage jobs and/or acceptance stage jobs after being deployed to some server), then the image is pushed one more time as the release image. This second push can be seen as the promotion of the snapshot image being now tested and reliable.

In practice:

  • the snapshot image is always pushed by the template (pipeline triggered by a Git tag or commit on any branch),
  • the release image is only pushed:
    • on a pipeline triggered by a Git tag,
    • on a pipeline triggered by a Git commit on master.

The snapshot and release images are defined by the following variables:

Input / Variable Description Default value
snapshot-image / S2I_SNAPSHOT_IMAGE S2I snapshot image $CI_REGISTRY_IMAGE/snapshot:$CI_COMMIT_REF_SLUG
release-image / S2I_RELEASE_IMAGE S2I release image $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME

As you can see, the S2I template is configured by default to use the GitLab container registry. You may perfectly override this and use another container registry, but be aware of a few things:

  • the S2I_SNAPSHOT_IMAGE requires a container registry that allows tag overwrite,
  • the S2I_RELEASE_IMAGE may use a container registry that doesn't allow tag overwrite, but:
    1. you should avoid overwriting a Git tag (at it will obviously fail while trying to (re)push the image),
    2. you have to deactivate publish on main (or master) branch by setting the $S2I_PROD_PUBLISH_STRATEGY variable to none (as it would lead to the main tag being overwritten).

Registries and credentials

As seen in the previous chapter, the S2I template uses by default the GitLab registry to push snapshot and release images. Thus it makes use of credentials provided by GitLab itself to login (CI_REGISTRY_USER / CI_REGISTRY_PASSWORD).

But when using other registry(ies), you'll have also to configure appropriate credentials.

Using the same registry for snapshot and release

If you use the same registry for both snapshot and release images, you shall use the following configuration variables:

Input / Variable Description
🔒 S2I_REGISTRY_USER container registry username for image registry
🔒 S2I_REGISTRY_PASSWORD container registry password for image registry

Using different registries for snapshot and release

If you use different registries for snapshot and release images, you shall use separate configuration variables:

Input / Variable Description
🔒 S2I_REGISTRY_SNAPSHOT_USER container registry username for snapshot image registry
🔒 S2I_REGISTRY_SNAPSHOT_PASSWORD container registry password for snapshot image registry
🔒 S2I_REGISTRY_RELEASE_USER container registry username for release image registry
🔒 S2I_REGISTRY_RELEASE_PASSWORD container registry password for release image registry

Secrets management

Here are some advices about your secrets (variables marked with a 🔒):

  1. Manage them as project or group CI/CD variables:
    • masked to prevent them from being inadvertently displayed in your job logs,
    • protected if you want to secure some secrets you don't want everyone in the project to have access to (for instance production secrets).
  2. In case a secret contains characters that prevent it from being masked, simply define its value as the Base64 encoded value prefixed with @b64@: it will then be possible to mask it and the template will automatically decode it prior to using it.
  3. Don't forget to escape special characters (ex: $ -> $$).

Jobs

s2i-build job

This job builds the image and publishes it to the snapshot repository.

It is bound to the package-build stage, and uses the following variables:

It uses the following variable:

Input / Variable Description Default value
root-dir / S2I_ROOT_DIR Relative path to the application source code base directory in your repository .
builder-image / S2I_BUILDER_IMAGE The S2I builder image used to build your application image none (required)
build-extra-flags / S2I_BUILD_EXTRA_FLAGS S2I build extra flags none

This job produces output variables that are propagated to downstream jobs (using dotenv artifacts):

Input / Variable Description Example
s2i_image snapshot image name with tag registry.gitlab.com/acme/website/snapshot:main
s2i_image_digest snapshot image name with digest (no tag) registry.gitlab.com/acme/website/snapshot@sha256:b7914a91...
s2i_repository snapshot image bare repository (no tag nor digest) registry.gitlab.com/acme/website/snapshot
s2i_tag snapshot image tag main
s2i_digest snapshot image digest sha256:b7914a91...

They may be freely used in downstream jobs (for instance to deploy the upstream built image, whatever the branch or tag).

s2i-publish job

This job pushes (promotes) the built image as the release image using skopeo.

Input / Variable Description Default value
publish-args / S2I_PUBLISH_ARGS Additional skopeo copy arguments (none)
prod-publish-strategy / S2I_PROD_PUBLISH_STRATEGY Defines the publish to production strategy. One of manual (i.e. one-click), auto or none (disabled). manual
release-extra-tags-pattern / S2I_RELEASE_EXTRA_TAGS_PATTERN Defines the image tag pattern that $S2I_RELEASE_IMAGE should match to push extra tags (supports capturing groups - see below) ^v?(?P<major>[0-9]+)\\.(?P<minor>[0-9]+)\\.(?P<patch>[0-9]+)(?P<suffix>(?P<prerelease>-[0-9A-Za-z-\\.]+)?(?P<build>\\+[0-9A-Za-z-\\.]+)?)$ (SemVer pattern)
release-extra-tags / S2I_RELEASE_EXTRA_TAGS Defines extra tags to publish the release image (supports capturing group references from $S2I_RELEASE_EXTRA_TAGS_PATTERN - see below) (none)
semrel-release-disabled / S2I_SEMREL_RELEASE_DISABLED Set to true to disable semantic-release integration none (enabled)

This job produces output variables that are propagated to downstream jobs (using dotenv artifacts):

Input / Variable Description Example
s2i_image release image name with tag registry.gitlab.com/acme/website:main
s2i_image_digest release image name with digest (no tag) registry.gitlab.com/acme/website@sha256:b7914a91...
s2i_repository release image bare repository (no tag nor digest) registry.gitlab.com/acme/website
s2i_tag release image tag main
s2i_digest release image digest sha256:b7914a91...

They may be freely used in downstream jobs (for instance to deploy the upstream built image, whatever the branch or tag).

Using extra tags

When publishing the release image, the S2I template might publish it again with additional tags (aliases):

  • the original published image tag (extracted from $S2I_RELEASE_IMAGE) must match $S2I_RELEASE_EXTRA_TAGS_PATTERN (semantic versioning pattern by default),
  • extra tags to publish can be defined in the $S2I_RELEASE_EXTRA_TAGS variable, each separated with a whitespace.

ℹ the S2I template supports group references substitution to evaluate extra tags:

  • $S2I_RELEASE_EXTRA_TAGS_PATTERN supports capturing groups:
    • v([0-9]+)\.([0-9]+)\.([0-9]+) has 3 (unnamed) capturing groups, each capturing any number of digits
    • v(?<major>[0-9]+)\.(?<minor>[0-9]+)\.(?<patch>[0-9]+) has 3 named capturing groups (major, minor and patch), each capturing any number of digits
  • $S2I_RELEASE_EXTRA_TAGS supports capturing group references from $S2I_RELEASE_EXTRA_TAGS_PATTERN:
    • \g1 is a reference to capturing group number 1
    • \g<major> is a reference to capturing group named major

ℹ the default value of $S2I_RELEASE_EXTRA_TAGS_PATTERN matches and captures all parts of a standard semantic versioning-compliant tag:

  • the major group captures the major version
  • the minor group captures the minor version
  • the patch group captures the patch version
  • the prerelease group captures the (optional) pre-release version (including the leading -)
  • the build group captures the (optional) build version (including the leading +)
  • the suffix group captures the (optional) entire suffix (including pre-release and/or build)

Example: publish latest, major.minor and major aliases for a SemVer release:

variables:
  # ⚠ don't forget to escape backslash character in yaml
  S2I_RELEASE_EXTRA_TAGS: "latest \\g<major>.\\g<minor>\\g<build> \\g<major>\\g<build>"

With this contiguration, the following extra tags would be published:

original tag extra tags
main none (doesn't match $S2I_RELEASE_EXTRA_TAGS_PATTERN)
some-manual-tag none (doesn't match $S2I_RELEASE_EXTRA_TAGS_PATTERN)
1.2.3 latest, 1.2, 1
1.2.3-alpha.12 latest, 1.2, 1
1.2.3+linux latest, 1.2+linux, 1+linux
1.2.3-alpha.12+linux latest, 1.2+linux, 1+linux

semantic-release integration

If you activate the semantic-release-info job from the semantic-release template, the s2i-publish job will rely on the generated next version info.

  • the release will only be performed if a semantic release is present
  • the tag will be based on SEMREL_INFO_NEXT_VERSION, it will override S2I_RELEASE_IMAGE by simply substituting the tag, or adding a tag when there's none.

For instance, in both cases:

S2I_RELEASE_IMAGE: "registry.gitlab.com/$CI_PROJECT_NAME"
S2I_RELEASE_IMAGE: "registry.gitlab.com/$CI_PROJECT_NAME:$CI_COMMIT_REF_NAME"
The published container image will be registry.gitlab.com/$CI_PROJECT_NAME:$SEMREL_INFO_NEXT_VERSION and all subsequent jobs relying on the s2i_image variable will be provided with this tag.

⚠ When semantic-release detects no release (i.e. either the semantic-release template is misconfigured, or there were simply no feat/fix commits), the s2i-publish job will report a warning and no image will be pushed in the release registry. In such a case, the s2i_image remains unchanged, and will refer to the snapshot image. Any subsequent job that may deploy to production (Kubernetes or Openshift), should thus be configured not to deploy in this situation. Refer to deployment template for more information.

Finally, the semantic-release integration can be disabled with the S2I_SEMREL_RELEASE_DISABLED variable.