Home Approach Services Mission Team Careers Blog Contact

Building Ansible Collections

6 minute read Published: Author: Derek Laventure
Cloud , Infrastructure , Ansible , Galaxy

The evolution of simplicity

Initially our pure ansible infrastructure project consisted primarily of Ansible playbooks, using includes and imports to segment out what was essentially one long sequential process. We built up a set of plays that would go from nothing through all of the support infrastructure and provisioning a Droplet, up to the point of being ready to install application-level software, namely Aegir and its dependencies.

At this point, we were able to pick up and dust off some roles that we had built over the last number of years while supporting clients with Aegir installations. The 2 key roles here were consensus.aegir and consensus.aegir-minion, and while updating and refreshing these for current use, were able to further simplify things, eliminating a number of custom forks of upstream roles etc. At the same time, we updated the publishing of these roles to align with the new best practice of rolling them into collections (deprecating and archiving the originals).

Having gone this far, we realized it would be straightforward to go one step further and roll our consensus.droplet role into a collection we could publish as well. This is arguably overkill in terms of simplicity, as we could have moved things into custom roles within the project quite easily. Two reasons convinced us to keep going.

First, our commitment to open source: anywhere possible, we try to publish and share our work in hopes that it can benefit others. Second, sometimes we help our future selves by properly documenting and publishing things in this way. In this case, having 2 different IaC repositories means that if we stabilize and externalize the custom code within each, we have less to manually re-implement: we can share code outside the IaC repos themselves.

Introducing our new Ansible Collections

Without further ado, let me introduce our shiny new Ansible collections:

These collections form the basis for the rest of our pure ansible infrastructure solution. Each collection contains a couple of roles, so let’s review each in turn.

consensus.infra: Droplets and administrators

As alluded to previously, this collection contains a role that was born of our initial work building linear playbooks to:

  • create a DigitalOcean Droplet,
  • provision a Block Storage Volume,
  • install a DNS A record at Gandi.net,
  • join our Tailscale VPN,
  • configure outbound email,
  • install standard OS packages (Consensus flavour),
  • restrict SSH access to the tailnet, and
  • create administrative users with SSH access to the droplet.

In fact, this last step is covered by the other role in this collection, admin-users. This role has been around for a number of years, and simply takes a list of users and SSH public keys, then creates those users on the target host, ensuring they have a .authorized_keys file to allow them to log in with their SSH key. Because these two are so closely linked for our purposes, we packaged them together into one collection.

While these roles are published and available for anyone to use, I will note that the consensus.infra.droplet role is still very fresh, and thus very opinionated. The set of steps I outline above is configurable in that you can set a custom hostname and domain, and so on. However, you can’t skip any steps, or change out your DNS provider, for example. If your needs don’t fit exactly the steps this role provides, you’ll need to adapt it to your needs. Feel free to submit an MR on the project or fork your own copy to customize!

consensus.aegir: Aegir3 and remote servers

Our second collection provisions the Aegir3 hosting system. As mentioned, these roles have been around a lot longer, and only needed some updates (for Ansible syntax, and newer OS releases) to be useful. We were also able to eliminate forks of a number of geerlingguy roles that we had previously customized to accommodate Aegir’s needs. Many of these customizations were no longer needed or incorporated upstream. We found ways to move those that remained directly into the Aegir role. This means we can now use only on the stock geerlingguy roles, and everything works fine.

A small caveat: in our focus on simplicity, we’ve narrowed the scope to exactly the OS and version we’re targeting at present, namely Ubuntu 24.04. We know these roles work well in that context, and should be reasonably functional for 22.04 and even 20.04, with the possible need for some tweaking of PHP versions.

If you’re targeting a Debian-like system, things are likely to work more or less as-is, but other Linux distributions are unlikely to function out of the box. That said, adapting to different distros and versions is usually a matter of adding some conditional flags that check the relevant Facts on the target host, and switch out the play they run appropriately. As always, we welcome patches!

If you get this role running in a new environment, please submit an MR on the project to share your work :)

Bonus: auto-publishing collections

Since we had now found ourselves with not one but two Ansible collections we wanted to publish (and more we can imagine wanting in future), we knew it would be wise to automate publishing to Galaxy. With the solo roles previously, our publishing workflow had been automatic but somewhat opaque and fragile. We prefer to work in GitLab, but Galaxy can only publish roles via GitHub. So we had set up mirroring of the repository to GitHub, and thus a push to the main branch in GitLab would result in a new package on Galaxy.

With the move to Collections, it’s possible to publish directly to Galaxy using a couple of commands on the shell. This meant we could eliminate the GitHub mirror, which was the fragile part, and implement a CI pipeline on GitLab that would push out new release packages when we set a new git tag on the main branch.

After some research, I found a post by Jeff Geerling himself describing the process he uses for this. I also found a couple of other posts that helped me adapt this to a GitLab context, and ended up producing a template project with the minimal pieces in place to auto-publish an Ansible Collection to Galaxy:

  • publish/publish-collection.yml: an Ansible playbook to build and publish the collection.
  • .gitlab-ci.yml: calls the playbook if: there is a git tag on the commit
  • publish/galaxy.yml.j2: template metadata file for your collection
  • meta/runtime.yml: boilerplate metadata
  • README: detailed instructions on how to adapt the template for your use :)

Final steps: pulling it all together

We have now reviewed all the major components of our pure ansible infrastructure project, and we’re ready to pull it all together. In our next post we’ll take a look at the final pieces, and take a step back to review the bigger, simpler picture.

The article Building Ansible Collections first appeared on the Consensus Enterprises blog.

We've disabled blog comments to prevent spam, but if you have questions or comments about this post, get in touch!