Automated testing ansible playbooks using gitlab-ci and molecule

When you are developing ansible playbooks you will come to the point when you have to (or want to) automate testing of the playbooks/roles before getting them to the production. Or even you would like to setup CI/CD system that plays your playbook against your production environment automatically, but of course previously tested them. Just follow this blog entry and I will try to show you in couple of simple steps how to fully automate testing ansible playbooks using gitlab-ci pipeline and molecule.

Contents list:

• What is molecule
• Installing gitlab-ci runner and docker components
• Adding molecule tests to your roles
  • Init new molecule role
  • Add molecule config to the existing role
• Configuring gitlab runner (.gitlab-ci.yml file)
• Summary

What is molecule

From molecule’s webpage:

Molecule is designed to aid in the development and testing of Ansible roles.

Molecule provides support for testing with multiple instances, operating systems and distributions, virtualization providers, test frameworks and testing scenarios.

Molecule encourages an approach that results in consistently developed roles that are well-written, easily understood and maintained.

Why to use molecule instead of just ansible-linter, ansible syntax checking and yamllinter? Because with molecule you can spin up a docker image with a given operating system and test playbooks against it. It will also do linting, syntax checking but it will also take care of checking if the playbooks/roles are idempotent. What is more you can write your own scripts so that they will be executed to test if the playbooks were run properly and if the system is configured in the way you wanted to. In my opinion just a basic tests that are run with molecule are enough for the beginning. I will try to include more complex tests in the next articles. So please stay tuned.

Installing gitlab-ci runner and docker components

In order to test ansible playbooks you will have to install gitlab-ci runner and docker components on the hosts that will be running all the tests. The simplest way to install and register gitlab runner is to follow the instructions on the gitlab’s webpage. Please be sure to use shell executor as other types of gitlab runners are not suitable to run docker containers (I know there is docker-machine and docker executor, but we will not be able to run docker images inside docker environment, so we have to run docker images on top of the operating system).

The next stop is to install docker components on your freshly installed and registered gitlab runner. To install docker please just visit docker’s webpage and follow instructions.

I am using CentOS 7 host as a gitlab runner and after installing above software you will not be able to run gitlab runner with molecule out of the box. To be able to tests you have to fix/install some dependencies.

First of all python-six package is quite out of date in the CentOS 7 installation so you have to update it manually:

pip install six --upgrade

The second thing is that you have to install docker python bindings so that molecule can spin up new docker containers:

pip install docker-py

After that you have fully working gitlab runner that is able to run molecule tests. Of course you can configure above things in the VM, which would be nicer than messing with a system’s python libriaries, or even use virtualenv to run tests (idea for next blog post?:) ).

Adding molecule tests to your roles

Hopefully you’ve followed official way to install molecule on the host (pip or in virtualenv), otherwise you will end up with old molecule installation (version 1.x) which is not compatible with newest version. All the setup and configurations steps that are presented here are done on the newest available version (at the time being 2.19.0).

To start testing we have to either init new role with molecule witihin your ansible setup or add molecule config to existing role.

Init new molecule role

If you are creating new role from scratch you can use:

molecule init --role my-new-role --driver docker

Where my-new-role is the name of the new role and docker is a driver that we will be using. It will also create directories for tasks/vars/handlers and it will set up basic config file for the molecule testing.

Add molecule config to existing role

If you already have a role and just want to add molecule testing for it please use the following command:

molecule init scenario --role my-role-name --driver docker

It will set up molecule testing files for your my-role-name with a docker driver (as we are going to use docker containers to test our playbooks/roles).

Configuring molecule

After we’ve added basic config files for the roles in the above section, we have to configure molecule. Whole configuration is of molecule tests are done in the molecule.yml file (for example in ansible/playbooks/roles/my-role-name/molecule/molecule.yml). Here is an example of configuration file (not original one, it’s modified by me):

---
dependency:
  name: galaxy
driver:
  name: docker
lint:
  name: yamllint
platforms:
  - name: centos7
    image: centos/systemd
    privileged: true
    command: /sbin/init
    security_opts:
      - seccomp=unconfigured
    capabilities:
      - SYS_ADMIN
    volume_mounts:
      - "/sys/fs/cgroup:/sys/fs/cgroup:ro"
provisioner:
  name: ansible
  lint:
    name: ansible-lint
scenario:
  name: default

The most important for sections are:

driver: as you can see the driver is set up to use docker for testing playbooks on

platorms: in this section we can set up as many platforms as we wish molecule to test on. 

In the above example I am setting up systemd enabled CentOS 7 docker image (that’s why there are a lot options under centos7 section) to test my roles against. Of course you can add another platforms like: CentOS6, Ubuntu (any version) or Debian. It all depends on your roles and against what would like to test them.

Basically with this setup we can run molecule manually to test our role:

cd ansible/playbooks/roles/my-role-name
molecule test

After invoking above command, molecule will do:

1. linting all the *.yml files found in the given location/role using following techniques:
   1. yamllint 
   2. flake8
   3. ansible-lint
2. try to destroy all orphaned docker images that could potentially interrupt with tests
3. try to cope with all the dependencies using ansible-galaxy
4. check syntax
5. prepare docker images
6. converge docker images using given roles
7. test idempotence of the given role
8. execute verification scripts from molecule/default/tests/
9. destroy all created docker images

If all tests passed without any issues then your role is written correctly and can be considered as production ready. 

Configuring gitlab runner (.gitlab-ci.yml file)

To execute molecule tests automatically when new changes are pushed to the git repo on our gitlab server we have to configure gitlab runner. It is as simple as just putting one file in the main directory of your repository. The file should be named .gitlab-ci.yml.

Here is an example of the gitlab runner configuration file:

---
variables:
GIT_DEPTH: "1"
PIP_CACHE_DIR: "$CI_PROJECT_DIR/pip-cache"

cache:
paths:
- "$CI_PROJECT_DIR/pip-cache"
key: "$CI_PROJECT_ID"

stages:
- molecule

molecule:
stage: molecule
tags:
- shell
script:
- cd playbooks/roles/basic/ && molecule test

Most important setting/sections are:

stages: we are saying what stages we are going to have in our pipeline. In this case just testing using molecule, but we could add more stages to it (every stage is run after preceding one has been succesful, so stages can be used to play role against production servers)

molecule: this describes and configures the molecule stage. The most important subsection is script, the rest could be ommited. Script is just a set of commands used to run tests in this stage.

All other setting are not so important for our purpose and you can read about them here

Summary

After following all the configuration steps your gitlab pipeline should be configured properly so after pushing new changes to git repo you should see green tick in the web gui near your latest commit, or red cross if the tests didn’t pass.

I’ve prepared small repository on github showing all the functionality described here. The repo can be found here

Please feel free to clone/comment/play with it. 

P.S. Hopefully this entry would be helpful. I have some ideas for the next blog entries so stay tuned. VY 73!