Introduction to the Introduction
Over the last few years we’ve built lots of Drupal 8 sites, and some Drupal 9 ones too, both for our clients and for ourselves. As such, we’ve taken a keen interest in (read: faced many challenges with) the Configuration Management subsystem. This was a major new component in Drupal 8, and so, while it’s functional, it isn’t yet mature. Of course, the vibrant Drupal developer community jumped in to smooth the rough edges and fill the gaps, in what has since become known as CMI 2.0.
At Consensus, we tend to work on fairly large, complex Drupal projects that often require significant custom development. As such, we’ve adopted fairly rigourous software engineering processes, such as standardized local development environments, CI-enabled Test Driven Development, Continuous Delivery, etc.
However, we struggled to find a workflow that leveraged the powerful CMI system in core, while not being a pain for developers.
Configuration Management in D8+
The core CMI workflow assumes you are transferring configuration between a single site install with multiple instances (Dev, Stage, Prod) and that this configuration is periodically synchronized as a lump. For a number of reasons, this wouldn’t work for us.
As a result, we went back to our old standby, the venerable Features module, which worked reasonably well. Unfortunately, we found that it would sometimes handle dependencies between configuration objects poorly. On more than one occasion, this led to time-consuming debugging cycles.
So we switched to using Config Profile instead. However, reverting config changes was still manual, so we started using Config Update and the related Update Helper.
The Update Helper module, “offers supporting functionalities to make configuration updates easier.” Basically, when preparing for a release, Update Helper generates a special file, a “configuration update definition” (CUD). The CUD contains two values for each changed config. The first is the “current” value for a given configuration, as of the most recent release. The second is the new value to which you want to set that config.
These values are captured by first rolling back to the most recent release,
then installing the site, so that the value is in active config. Then you
checkout your latest commit, so that the new values are available on the
filesystem. Update Helper can then generate its CUD, as well as generate a
hook_update() implementations to help deploy the new or changed config.
This process turned out to be error-prone, and difficult to automate reliably.
We explored other efforts too, like Config Split and Config Filter which allow for finer-grained manipulation of “sets” of config. Other projects, like Config Distro, are focused on “packaging” groups of features such that they can be dropped in to any given site easily (kind of like Features…)
A simple, reliable method to deploy new or updated configuration remained elusive.
The underlying problem
Note that all the tools mentioned above work very well during initial project development, prior to production release. However, once you need to deploy config changes to systems in production Update Helper or similar tools and processes are required, along with all the overheads that implies.
At this point, it’s worth reminding ourselves that Drupal 7 and earlier versions did not clearly distinguish between content and config. They all just lived in the site’s database, after all. As such, whatever configuration was on the production site was generally considered canonical.
It’s tempting to make small changes directly in production, since they don’t seem to warrant a full release, and all the configuration deployment overhead that entails. This, in turn, requires additional discipline to reproduce those changes in the codebase.
Of course, that isn’t the only reason for configuration drift. Well-meaning administrators cannot easily distinguish between configs that are required for the proper operation of the site, and those that have more cosmetic effects.
Facing these challenges, we’d regularly note how much easier all of this would be if only we could make production configuration read-only.
A new approach
With some reluctance and much consideration, we decided to try an entirely new approach. We built Config Enforce (and its close companion Config Enforce Devel) to solve the two key problems we were running into:
- Developers needed an easy way to get from “I made a bunch of config-related changes in the Admin UI of my local site instance” to “I can identify the relevant config objects/entities which have changed, get them into my git repository, and push them upstream for deployment”.
- Operations needed an easy way to deploy changes in configuration, and ideally not have to worry too much about the previously-inevitable “drift” in the production-environment configuration, which often resulted in tedious and painful merging of configs, or worse yet, inadvertent clobbering of changes.
Config Enforce has two “modes” of operation: with config_enforce_devel
enabled, you are in “development mode”. You can quickly designate config
objects you want to enforce (usually inline on the configuration page where you
manipulate the configuration object itself), and then changes you make are
immediately written to the file system.
This mode leverages Config Devel to
effectively bypass the active configuration storage in the database, writing
the config files into target extensions you select. Each target extension
builds up a “registry” of enforced configuration objects, keeping track of
their location and enforcement level. This eases the development workflow by
making it easy to identify which configuration objects you’ve changed without
having to explictly config-export and then identify all and only the relevant
.yml files to commit.
In production mode, you enable only the config_enforce module, which
leverages the same “registry” configuration that config_enforce_devel has
written into your target extensions, and performs the “enforcement” component.
This means that, for any enforced configuration objects, we block any changes
from being made via the UI or optionally even API calls directly. In turn, the
enforced configuration settings on the file system within target extensions
become authoritative, being pulled in to override whatever is in active
configuration whenever a cache rebuild is triggered.
This means that deployment of configuration changes becomes trivial: commit and push new enforced configuration files in your target extensions, pull those into the new (e.g., Prod) environment, and clear caches. configuration Enforce will check all enforced configuration settings for changes on the file system, and immediately load them into active configuration on the site.
This workflow requires some adjustment in how we think about configuration management, but we think it has promise. Especially if you are building Drupal distributions or complex Drupal systems that require repeatable builds and comprehensive testing in CI, you should give Config Enforce a try and see what you think. Feedback is always welcome!
We’ve scratched our own itch, and so far have found it useful and productive. We are pleased to make it available to the Drupal community as another in the arena of ideas surrounding CMI 2.0.
The article Introducing Config Enforce first appeared on the Consensus Enterprises blog.
We've disabled blog comments to prevent spam, but if you have questions or comments about this post, get in touch!