Skip to content

GitLab CI template for Ansible

This project implements a Gitlab CI template for Ansible.

Overview: managed environments

This template implements Ansible-based deployments.

It allows you to manage automatic deployment & cleanup of standard predefined environments. Each environment can be enabled/disabled by configuration. If you're not satisfied with predefined environments and/or their associated Git workflow, you may implement you own environments and workflow, by reusing/extending the base (hidden) jobs. This is advanced usage and will not be covered by this documentation.

The following chapters present the managed predefined environments and their associated Git workflow.

Review environments

The template supports review environments: those are dynamic and ephemeral environments to deploy your ongoing developments (a.k.a. feature or topic branches).

When enabled, it deploys the result from upstream build stages to a dedicated and temporary environment. It is only active for non-production, non-integration branches.

It is a strict equivalent of GitLab's Review Apps feature.

It also comes with a cleanup job (accessible either from the environments page, or from the pipeline view).

Integration environment

If you're using a Git Workflow with an integration branch (such as Gitflow), the template supports an integration environment.

When enabled, it deploys the result from upstream build stages to a dedicated environment. It is only active for your integration branch (develop by default).

Production environments

Lastly, the template supports 2 environments associated to your production branch (master or main by default):

  • a staging environment (an iso-prod environment meant for testing and validation purpose),
  • the production environment.

You're free to enable whichever or both, and you can also choose your deployment-to-production policy:

  • continuous deployment: automatic deployment to production (when the upstream pipeline is successful),
  • continuous delivery: deployment to production can be triggered manually (when the upstream pipeline is successful).

Usage

Include

In order to include this template in your project, add the following to your gitlab-ci.yml:

include:
  - project: 'to-be-continuous/ansible'
    ref: '2.1.1'
    file: '/templates/gitlab-ci-ansible.yml'

variables:
  ANSIBLE_DEFAULT_INVENTORY: "inventory.ini"
  ANSIBLE_DEFAULT_TAGS: "install"
  ANSIBLE_DEFAULT_EXTRA_ARGS: "-b"
  ANSIBLE_PROD_PLAYBOOK_FILE: "main.yml"

Don't forget to update the variables values ;-)

Global configuration

This template can be configured with the following environment variables:

Name Description Default value
ANSIBLE_IMAGE The Docker image used to run Ansible. The image may contain your Ansible sources.
⚠ī¸ set the version required by your project
cytopia/ansible:latest-tools
ANSIBLE_PROJECT_DIR Ansible project root directory .
🔒 ANSIBLE_VAULT_PASSWORD The Ansible vault password used to decrypt vars. has to be defined in gitlab secret if used
🔒 ANSIBLE_PRIVATE_KEY The Ansible SSH private key to use in all stages (can be overridden per env) has to be defined in gitlab secret if used
ANSIBLE_PUBLIC_KEY The Ansible SSH public key associated to the private key to be use in all stages (can be overridden per env) has to be defined if used
ANSIBLE_DEFAULT_INVENTORY The default inventory, if used has to be defined if used
ANSIBLE_DEFAULT_TAGS The default tags, if used has to be defined if used
ANSIBLE_DEFAULT_EXTRA_ARGS Optional default args to add to the ansible-playbook command line has to be defined if used
ANSIBLE_DEFAULT_ROLES_PATH The default path where the roles should be installed $CI_PROJECT_DIR/roles
ANSIBLE_FORCE_COLOR Forces color on Ansible output true
ANSIBLE_REQUIREMENTS_FILE The file used to install roles with ansible-galaxy install requirements.yml
ANSIBLE_SCRIPTS_DIR The Ansible scripts base directory (relative to $ANSIBLE_PROJECT_DIR) .
ANSIBLE_HOST_KEY_CHECKING Enable or disable the SSH host key checking false (disabled)

Secrets management

Here are some advices about your secrets (variables marked with a 🔒):

  1. Manage them as project or group CI/CD variables:
    • masked to prevent them from being inadvertently displayed in your job logs,
    • protected if you want to secure some secrets you don't want everyone in the project to have access to (for instance production secrets).
  2. In case a secret contains characters that prevent it from being masked, simply define its value as the Base64 encoded value prefixed with @b64@: it will then be possible to mask it and the template will automatically decode it prior to using it.
  3. Don't forget to escape special characters (ex: $ -> $$).

Manage remote repositories authentication

Specifying an Ansible requirements file (with $ANSIBLE_REQUIREMENTS_FILE), you may use Ansible roles from remote places (Ansible repository, Git repository, ...). Accessing those repositories may require an authentication.

In such a case, you may configure the remote authentication(s) by providing a variabilized .netrc file at the root of your Ansible project. Here is an example:

machine gitlab.com login gitlab-ci-token password ${CI_JOB_TOKEN}
machine ansible.acme.example login ${ACME_ANSIBLE_USER} password ${ACME_ANSIBLE_PASSWORD}

When such a file is detected, the Ansible template replaces each ${ENV} patterns with actual environment values and installs it prior to executing ansible-galaxy install.

Environments configuration

As seen above, the Kubernetes template may support up to 4 environments (review, integration, staging and production).

Each deployment job produces output variables that are propagated to downstream jobs (using dotenv artifacts):

  • environment_type: set to the type of environment (review, integration, staging or production),
  • environment_url: set to $CI_ENVIRONMENT_URL.

They may be freely used in downstream jobs (for instance to run acceptance tests against the latest deployed environment).

Here are configuration details for each environment.

Review environments

Review environments are dynamic and ephemeral environments to deploy your ongoing developments (a.k.a. feature or topic branches).

It is disabled by default and can be enabled by setting the ANSIBLE_REVIEW_PLAYBOOK_FILE variable (see below).

Here are variables supported to configure the integration environment:

Name Description Default value
ANSIBLE_REVIEW_INVENTORY The inventory for review env $ANSIBLE_DEFAULT_INVENTORY
ANSIBLE_REVIEW_TAGS The tags for review env $ANSIBLE_DEFAULT_TAGS
ANSIBLE_REVIEW_CLEANUP_TAGS The tags to cleanup the review env has to be defined if used
ANSIBLE_REVIEW_EXTRA_ARGS The command line args for review env $ANSIBLE_DEFAULT_EXTRA_ARGS
ANSIBLE_REVIEW_PLAYBOOK_FILE The playbook filename for review env has to be defined to enable the review env
ANSIBLE_REVIEW_CLEANUP_PLAYBOOK_FILE The playbook filename to cleanup review env $ANSIBLE_REVIEW_PLAYBOOK_FILE
🔒 ANSIBLE_REVIEW_PRIVATE_KEY The SSH private key to be use in review env $ANSIBLE_PRIVATE_KEY
ANSIBLE_REVIEW_PUBLIC_KEY The SSH public key associated to the private key to be use in review env $ANSIBLE_PUBLIC_KEY
🔒 ANSIBLE_REVIEW_VAULT_PASSWORD The Ansible vault password for review env $ANSIBLE_VAULT_PASSWORD

Integration environment

The integration environment is the environment associated to your integration branch (develop by default).

It is disabled by default and can be enabled by setting the ANSIBLE_INTEG_PLAYBOOK_FILE variable (see below).

Here are variables supported to configure the integration environment:

Name Description Default value
ANSIBLE_INTEG_INVENTORY The inventory for integration env $ANSIBLE_DEFAULT_INVENTORY
ANSIBLE_INTEG_TAGS The tags for integration env $ANSIBLE_DEFAULT_TAGS
ANSIBLE_INTEG_CLEANUP_TAGS The tags to cleanup the integration env has to be defined if used
ANSIBLE_INTEG_EXTRA_ARGS The command line args for integration env $ANSIBLE_DEFAULT_EXTRA_ARGS
ANSIBLE_INTEG_PLAYBOOK_FILE The playbook filename for integration env has to be defined to enable the integration env
ANSIBLE_INTEG_CLEANUP_PLAYBOOK_FILE The playbook filename to cleanup integration env $ANSIBLE_INTEG_PLAYBOOK_FILE
🔒 ANSIBLE_INTEG_PRIVATE_KEY The SSH private key to be use in integration env $ANSIBLE_PRIVATE_KEY
ANSIBLE_INTEG_PUBLIC_KEY The SSH public key associated to the private key to be use in integration env $ANSIBLE_PUBLIC_KEY
🔒 ANSIBLE_INTEG_VAULT_PASSWORD The Ansible vault password for integration env $ANSIBLE_VAULT_PASSWORD

Staging environment

The staging environment is an iso-prod environment meant for testing and validation purpose associated to your production branch (master by default).

It is disabled by default and can be enabled by setting the ANSIBLE_STAGING_PLAYBOOK_FILE variable (see below).

Here are variables supported to configure the staging environment:

Name Description Default value
ANSIBLE_STAGING_INVENTORY The inventory for staging env $ANSIBLE_DEFAULT_INVENTORY
ANSIBLE_STAGING_TAGS The tags for staging env $ANSIBLE_DEFAULT_TAGS
ANSIBLE_STAGING_CLEANUP_TAGS The tags to cleanup the staging env has to be defined if used
ANSIBLE_STAGING_EXTRA_ARGS The command line args for staging env $ANSIBLE_DEFAULT_EXTRA_ARGS
ANSIBLE_STAGING_PLAYBOOK_FILE The playbook filename for staging env has to be defined to enable the staging env
ANSIBLE_STAGING_CLEANUP_PLAYBOOK_FILE The playbook filename to cleanup staging env $ANSIBLE_STAGING_PLAYBOOK_FILE
🔒 ANSIBLE_STAGING_PRIVATE_KEY The SSH private key to be use in staging env $ANSIBLE_PRIVATE_KEY
ANSIBLE_STAGING_PUBLIC_KEY The SSH public key associated to the private key to be use in staging env $ANSIBLE_PUBLIC_KEY
🔒 ANSIBLE_STAGING_VAULT_PASSWORD The Ansible vault password for staging env $ANSIBLE_VAULT_PASSWORD

Production environment

Name Description Default value
AUTODEPLOY_TO_PROD Set this variable to auto-deploy to production. If not set deployment to production will be manual (default behaviour). has to be defined to enable automatic prod deployment
ANSIBLE_PROD_INVENTORY The inventory for production env $ANSIBLE_DEFAULT_INVENTORY
ANSIBLE_PROD_TAGS The tags for production env $ANSIBLE_DEFAULT_TAGS
ANSIBLE_PROD_EXTRA_ARGS The command line args for production env $ANSIBLE_DEFAULT_EXTRA_ARGS
ANSIBLE_PROD_PLAYBOOK_FILE The playbook filename for production env has to be defined
🔒 ANSIBLE_PROD_PRIVATE_KEY The SSH private key to be use in production env $ANSIBLE_PRIVATE_KEY
ANSIBLE_PROD_PUBLIC_KEY The SSH public key associated to the private key to be use in production env $ANSIBLE_PUBLIC_KEY
🔒 ANSIBLE_PROD_VAULT_PASSWORD The Ansible vault password for production env $ANSIBLE_VAULT_PASSWORD

Variables that you want to define or override default have to be defined as a project or group CI/CD variable or globally in your .gitlab-ci.yml file.

SSH Keys

When you set ANSIBLE_xxx_PRIVATE_KEY and/or ANSIBLE_xxx_PUBLIC_KEY in your environment, you will be able to use the following variables:

  • ANSIBLE_SSH_PRIVATE_KEY_FILE: The path of the public key file (if ANSIBLE_xxx_PRIVATE_KEY is used)
  • ANSIBLE_SSH_PUBLIC_KEY_FILE: The path of the private key file (if ANSIBLE_xxx_PUBLIC_KEY is used)

Cleanup environments

To activate environment cleanup, you need to set ANSIBLE_xxx_CLEANUP_PLAYBOOK_FILE and/or ANSIBLE_xxx_CLEANUP_TAGS in your environment.

Hook scripts

Deployment jobs also support optional hook scripts from your project, located in the $ANSIBLE_SCRIPTS_DIR directory (root project dir by default, but may be overridden).

  • pre-ansible-playbook.sh is executed before running ansible-playbook, to perform specific environment pre-initialization such as generating a dynamic inventory
  • post-ansible-playbook.sh is executed after running ansible-playbook, to perform specific environment post-initialization

Static vs. Dynamic environment URLs

The Ansible template supports two ways of defining your environments url:

  • a static way: when you know your environments url in advance, probably because you're exposing your routes through a DNS you manage,
  • a dynamic way: when the url cannot be known before the deployment job is executed.

The static way can be implemented simply by setting the overridding the environment:url attribute in each deployment job.

Example:

ansible-staging:
  environment:
    url: "https://myapp-staging.noprod.domain"

ansible-production:
  environment:
    url: "https://myapp.prod.domain"

To implement the dynamic way, your post deployment hook script shall simply generate a environment_url.txt file, containing only the dynamically generated url. This will be considered by the template as the newly deployed environment url.

Ansible lint job

The Ansible template enables Ansible Lint analysis of your Ansible scripts.

This job is mapped to the test stage and is active by default.

Here are its parameters:

Name Description Default value
ANSIBLE_LINT_IMAGE The Docker image used to run Ansible Lint haxorof/ansible-lint:latest
ANSIBLE_LINT_DISABLED Set to true to disable the ansible lint analysis none (enabled)

Examples

  • Review environment enabled,
  • Continuous deployment to production.

Examples are available in the sample section

Deploy and configure your infrastructure components

The gitlab-ci in this example, will show use how to configure the template to deploy your infrastructure and configure your components, from your ansible playbook repository. If you don't use an inventory, for example if you're building an openstack infrastruture, you can remove the *_INVENTORY variables.

include:
  - project: 'to-be-continuous/ansible'
    ref: '2.1.1'
    file: '/templates/gitlab-ci-ansible.yml'

variables:
  # Default variables
  # In this example, I consider that you have one inventory per platform 
  # and only one playbook file for all environments
  ANSIBLE_DEFAULT_EXTRA_ARGS: "-b" 
  ANSIBLE_DEFAULT_TAGS: "install"

  # Review
  ANSIBLE_REVIEW_INVENTORY: "review.ini"
  ANSIBLE_REVIEW_CLEANUP_TAGS: "destroy"
  ANSIBLE_REVIEW_PLAYBOOK_FILE: "$ANSIBLE_PROD_PLAYBOOK_FILE"

  # Staging
  ANSIBLE_STAGING_INVENTORY: "staging.ini"
  ANSIBLE_STAGING_PLAYBOOK_FILE: "$ANSIBLE_PROD_PLAYBOOK_FILE"

  # Production
  ANSIBLE_PROD_INVENTORY: "production.ini"
  ANSIBLE_PROD_PLAYBOOK_FILE: "main.yml" 

stages:
  - build
  - deploy
  - production

Don't forget to add $ANSIBLE_VAULT_PASSWORD & $ANSIBLE_PRIVATE_KEY in Gitlab secret variables.

Deploy your application

The gitlab-ci in this example, will show use how to configure the template to deploy your application, from your app repository (different from the ansible repo).

Ansible playbook repository

To use this template from a gitlab project that doesn't contains you ansible playbook (for example, your code repository), you need to package your ansible sources. Add this Dockerfile in your ansible project and edit the ENV variables to match your needs:

FROM williamyeh/ansible:alpine3

# Add here every package you will need to build libraries
# These packages will be removed after dependancies install 
ENV BUILD_PACKAGES="git gcc libc-dev linux-headers python2-dev"
# Packages to include in the image
ENV PACKAGES="ca-certificates bash openssh curl"
# Python dependancies to include in the image
ENV PIP_PACKAGES="openstacksdk"
# Your ansible source folder
ENV ANSIBLE_SRC_DIR="."

# Create ansible group and user
RUN addgroup -S ansible && adduser -S ansible -G ansible

RUN apk add --no-cache --virtual .build-deps $BUILD_PACKAGES \
    && apk add --no-cache $PACKAGES \
    && pip install --no-cache-dir --upgrade pip \
    && pip install --no-cache-dir --upgrade $PIP_PACKAGES \
    && apk del --no-cache --purge .build-deps \
    && rm -rf /var/cache/apk/*

# Tell docker that all future commands should run as ansible user
USER ansible

COPY $ANSIBLE_SRC_DIR   /home/ansible/playbook

WORKDIR /home/ansible/playbook

ENTRYPOINT [""]
CMD ["ansible-playbook","--version"]

In the .gitlab-ci.yml add the docker template to build your image:

include:
  - project: 'to-be-continuous/ansible'
    ref: '2.1.1'
    file: '/templates/gitlab-ci-ansible.yml'
  - project: 'to-be-continuous/docker'
    ref: '2.1.1'
    file: '/templates/gitlab-ci-docker.yml'

variables:
  # Default variables
  # In this example, I consider that you have one inventory per platform 
  # and only one playbook file for all environments
  ANSIBLE_DEFAULT_EXTRA_ARGS: "-b"
  ANSIBLE_DEFAULT_TAGS: "install"

  # Review
  ANSIBLE_REVIEW_INVENTORY: "review.ini"
  ANSIBLE_REVIEW_CLEANUP_TAGS: "destroy"
  ANSIBLE_REVIEW_PLAYBOOK_FILE: "$ANSIBLE_PROD_PLAYBOOK_FILE"

  # Staging
  ANSIBLE_STAGING_INVENTORY: "staging.ini"
  ANSIBLE_STAGING_PLAYBOOK_FILE: "$ANSIBLE_PROD_PLAYBOOK_FILE"

  # Production
  ANSIBLE_PROD_INVENTORY: "production.ini"
  ANSIBLE_PROD_PLAYBOOK_FILE: "main.yml"  

stages:
  - build
  - package-build
  - package-test
  - deploy
  - publish
  - production

By default, the docker template will push the image in the Gitlab docker registry. Please visit, the docker template page to get more information on Docker Template

Application repository

Add this .gitlab-ci.yml in the repository of your application project (you will need to add your build & package configuration in this ci)

include:
  - project: 'to-be-continuous/ansible'
    ref: '2.1.1'
    file: '/templates/gitlab-ci-ansible.yml'

variables:
  # Default variables
  # In this example, I consider that you have one inventory per platform 
  # and only one playbook file for all environments
  ANSIBLE_DEFAULT_EXTRA_ARGS: "-b" 
  ANSIBLE_DEFAULT_TAGS: "deploy"
  # Ansible lint has been launched on the ansible pipeline
  ANSIBLE_LINT_DISABLED: "true"
  # The ansible image with your playbook
  ANSIBLE_IMAGE: "registry.example/ender/ansible-playbook:master"

  # Review
  ANSIBLE_REVIEW_INVENTORY: "/home/ansible/playbook/review.ini"
  ANSIBLE_REVIEW_CLEANUP_TAGS: "undeploy"
  ANSIBLE_REVIEW_PLAYBOOK_FILE: "$ANSIBLE_PROD_PLAYBOOK_FILE"

  # Staging
  ANSIBLE_STAGING_INVENTORY: "/home/ansible/playbook/staging.ini"
  ANSIBLE_STAGING_PLAYBOOK_FILE: "$ANSIBLE_PROD_PLAYBOOK_FILE"

  # Production
  ANSIBLE_PROD_INVENTORY: "/home/ansible/playbook/production.ini"
  ANSIBLE_PROD_PLAYBOOK_FILE: "/home/ansible/playbook/main.yml" 

stages:
  - build
  - deploy
  - production

Variants

The default Ansible template is designed to work on untagged runners, without any proxy configuration using Docker images from the internet and using Default Trusted Certificate Authorities.

Nevertheless there are template variants available to cover specific cases.

Vault variant

This variant allows delegating your secrets management to a Vault server.

Configuration

In order to be able to communicate with the Vault server, the variant requires the additional configuration parameters:

Name description default value
VAULT_BASE_URL The Vault server base API url none
🔒 VAULT_ROLE_ID The AppRole RoleID must be defined
🔒 VAULT_SECRET_ID The AppRole SecretID must be defined

Usage

Then you may retrieve any of your secret(s) from Vault using the following syntax:

@url@http://vault-secrets-provider/api/secrets/{secret_path}?field={field}

With:

Name description
secret_path (path parameter) this is your secret location in the Vault server
field (query parameter) parameter to access a single basic field from the secret JSON payload

Example

include:
  # main template
  - project: 'to-be-continuous/ansible'
    ref: '2.1.1'
    file: '/templates/gitlab-ci-ansible.yml'
  # Vault variant
  - project: 'to-be-continuous/ansible'
    ref: '2.1.1'
    file: '/templates/gitlab-ci-ansible-vault.yml'

variables:
    # Secrets managed by Vault
    ANSIBLE_VAULT_PASSWORD: "@url@http://vault-secrets-provider/api/secrets/b7ecb6ebabc231/my-infra/ansible?field=vault.password"
    ANSIBLE_PRIVATE_KEY: "@url@http://vault-secrets-provider/api/secrets/b7ecb6ebabc231/my-infra/ansible?field=priv_key"
    VAULT_BASE_URL: "https://vault.acme.host/v1"
    # $VAULT_ROLE_ID and $VAULT_SECRET_ID defined as a secret CI/CD variable

GitLab compatibility

ℹī¸ This template is actually tested and validated on GitLab Community Edition instance version 13.12.11