You are reading our blog series: Pure Ansible Infrastructure
- Toward infrastructure simplicity
- Drumkit and other plumbing (You are here)
- Dynamic inventory
- Variables and Vault
- Playbooks and support infrastructure
- Building Ansible Collections
- make targets, Droplets, and Aegir, oh my!
Drumkit
Drumkit is one of my favourite tools that we use at Consensus, because it serves to unify and simplify how we interact with our project tooling. Drumkit itself is very simple, leveraging the venerable GNU make to create short, project-specific “targets” which are essentially a list of sub-commands to run. Any time we have a complex command-line command to do something, and we need to run it regularly, we add a Drumkit target for it and no longer have to remember the correct flags to consistently do the same thing.
In the case of our example Ansible infrastructure project, we leverage Drumkit for pinning a project-local installation of Ansible (and other tools) at a specific version, but primarily to provide easy access to key operations on the project itself:
make tools
(drumkit/mk.d/20_infra.mk - ensure we have Ansible, as well as the Galaxy requirements inrequirements.yml
make infra
(drumkit/mk.d/20_infra.mk - run the playbook to create a Project, VPC and Firewall at DigitalOceanmake aegir
(drumkit/mk.d/30_aegir.mk - combine the 2 following targets to create and configure the Aegir dropletmake aegir-droplet
- run the playbook to spin up a new dropletmake aegir-app
- run the playbook to configure Aegir and its dependencies
make aegir-down
- deregister the server from the VPN and tear down its resources
Installing Drumkit is easy enough, and using
it is as simple as sourcing the bootstrap
script
in .mk/scripts/bootstrap.sh
. Drumkit installs a symlink named d
at the top
of your project, so you can simply source d
when you first move into the
project, and Drumkit will set up your PATH to point to project-local tools, and
run any scripts in drumkit/bootstrap.d/
.
The Drumkit installer creates a number of files and directories you can commit into the repository:
.gitmodules
&.mk
- this is Drumkit itself,git submodule'd
into the project. This is also where your project-local binaries go (.mk/.local/bin
)d
-bootstrap.sh
symlinkdrumkit/bootstrap.d/*.sh
- an initial set of simple boostrap shell scripts triggered bysource d
Makefile
- the top-level Makefile, which simply includes Drumkit’s.mk/GNUMakefile
, that in turn includes the rest of Drumkit and your project’smake
target definitions indrumkit/mk.d/*.mk
.drumkit/mk.d/
- the directory where your project-local Makefiles (.mk
) will live. I typically create a10_variables.mk
in this file shortly after installing Drumkit to set the versions of tools required.
The bootstrap scripts checks for the environment variable DRUMKIT=1
to
determine if Drumkit is already bootstrapped in this shell, and then sets that
flag if so. It then adjusts your PATH
to include the project-local
.mk/.local/bin
where any tools (like ansible
) will be located after you run
a make ansible
or similar.
NB Git submodules
are a powerful way to integrate different git repositories, but they do require
that you subsequently clone your repository with the --recursive
tree every
time, in order to also clone the submodules into a new working tree. If you
forget this step, you can do git submodule update --init --recursive
to fix
it :)
Python
Ansible is a Python-based tool, and so in addition to the basic Drumkit tooling, we need a basic Python environment within which to operate.
Let me say up front, I’m not a very skilled Pythonista. I speak a number of programming languages with varying degress of fluency, and I know enough that I would count Python among them. However, I’ve never been immersed enough in the community and culture of Python to really grok it. I am therefore conscious that I may be missing something entirely in the foregoing section, and hoping some savvy Python pixie will drop me a line to say “here, you can just do it like this” ;)
One thing I do understand about the modern Python is that it really wants you
to do everything in a virtualenv. PEP 668: Marking Python base environments
as externally managed is an example of this
trend. For a Python-focused project using Drumkit, this presents a challenge
with respect to bootstrapping. We need to activate a python virtualenv
(essentially a sub-shell) and have Drumkit manipulate the shell environment.
If we bootstrap Drumkit first, then its environment gets clobbered when we
activate the virtualenv. However, I couldn’t find a way to “chain” the two
together, so I could activate the virtualenv and have it immediately trigger a
source d
. Even better, I’d love to drop a
drumkit/bootstrap.d/00_virtualenv.sh
which would do the virtualenv setup as
part of a regular source d
.
What I’ve landed on is using pipenv for its simplicity and ability to manage dependencies and versions effectively. With a simple Pipfile in place to register the python library requirements Ansible will need, we can get started in the project like so:
pipenv shell # Activate virtualenv
. d # Bootstrap Drumkit
make tools # Install ansible and galaxy requirements
Note the source d
includes a pipenv install
from
drumkit/bootstrap.d/10_pipenv.sh
to actually install the Pipfile requirements into the virtualenv.
Plumbing
As a developer, streamlining my workflows and making my daily tools quick and comfortable to use is essential to have the “plumbing” of my project just get out of the way, and let me focus on the problem at hand.
With this Drumkit and Python virtualenv setup in place, I have a shared set of versions and tailored-commands for common operations in the project, and I can extend it readily.
Having done the bootstrapping steps above, I can run any Ansible tool like
ansible-galaxy
or ansible-inventory
, or equally any of the
currently-available Drumkit targets. As I build out more pieces of the project,
I implement new Drumkit targets in simple modules under drumkit/mk.d
to
provide easy “wrappers” around commands. Thus, instead of remembering to type
ansible-playbook playbooks/hosts/aegir0.yml && ansible-playbook playbooks/apps/aegir.yml
, I can simply run make aegir
.
We now have the foundation of our Ansible project in place. In the next entry in this series, we’ll build on this tooling to implement Ansible dynamic inventory.
The article Drumkit and other plumbing first appeared on the Consensus Enterprises blog.
We've disabled blog comments to prevent spam, but if you have questions or comments about this post, get in touch!