Puppet or Chef: The configuration management dilemma
- 27 March, 2013 10:14
For the last few years, the game plan around the office was simple. The CEO would say "faster" and the CTO would say "cloud." Everyone would nod and say, "Of course."
All good things come to an end. The word "cloud" used to bring a positive, happy vibe to the enterprise staff because it promised to lift so many chores from our shoulders. The problems were going to melt away after an application of magic cloud ointment dispensed with a tap of the cloud wand on a Web page.
[ Stay on top of the cloud with InfoWorld's "Cloud Computing Deep Dive" special report. Download it today! | From Amazon to Windows Azure, see how the elite 8 public clouds compare in InfoWorld Test Center's review. | For the latest news and happenings, subscribe to the Cloud Computing Report newsletter. ]
Alas, everyone is discovering the cloud has plenty of its own headaches to replace all of the old headaches it solves. The trouble is that we're now responsible for dozens, hundreds, perhaps even thousands of machines. The cloud made it easy for us to deploy them, but now we have to take care of them. It's like that old maxim about kittens and stray dogs: Feed them once and you'll take care of them forever.
In this case, the joy of deploying dozens, hundreds, and perhaps thousands of machines means that there will soon be dozens, hundreds, perhaps even thousands of patches to apply, security holes to fix, and updates to download. When we had only a few machines, this kind of maintenance took a few minutes. But multiplying those minutes by dozens, hundreds, or thousands produces chores measured in days, weeks, and months.
Thank goodness for automation. Over the years, smart sys admins looked at the ballooning task list and figured out a way to write scripts that would handle the repetitive tasks. They built their own junior robot sys admin to do the work for them.
The hard work has coalesced into two major factions called Puppet and Chef. There are a number of other notable projects with readable names like Ansible and unreadable names like Bcfg2, but Puppet and Chef seem to have gathered the most excitement for now.
Both are open source stacks of code designed to make it easy for you to reach out and touch the files in your vast empire of virtual machines. Both have open source marketplaces for you to swap plug-ins that extend the framework and handle your particular type of hardware or software. Both are pretty cool, and both are finding homes in the racks of data centers around the world. Both now have companies built around the open source core selling assistance.
Models vs. proceduresAre there differences? Not in the most abstract sense. They both send instructions to configure or reconfigure your collection of machines. And "reconfigure" generally means installing new versions of libraries or other files.
But the world is full of factions, and whenever we get close to a unifying harmonic convergence, humans split into competing groups. Humans find some way to splinter.
In this case the fight is over language. Chef is written in Ruby and Erlang, and you'll get to write your specifications or extensions in Pure Ruby, a subset of the full and apparently impure language that is sometimes called a domain-specific langauge. This subset will give you just enough rope to hang yourself.
Puppet, on the other hand, has its very own language that bundles together all of the installation requirements into packages with curly brackets. It's a bit like JSON, but with conditional operators and object-oriented class structure added for fun. If you don't like it, Puppet started letting you write your specifications in Ruby after version 2.6.
It's probably not a good idea to spend too much time worrying about the language -- the real differences lie deeper. Puppet code is a list of dependencies that Puppet interprets. You might say, "Install A before B," and you could say this anywhere in your code. Puppet will find it and ensure that A is added first.
Chef, on the other hand, is more transparent. You better make sure the section for installing A comes before the section for installing B because Chef follows the instructions it's given.
Which is right for you? Luckily there are people writing long articles arguing for their vision. One that I liked is Steve Traugott and Lance Brown's "Why Order Matters: Turing Equivalence in Automated Systems Administration." Chef lovers like how it says that all of those dependencies turn into a crazy mess that doesn't encourage coherence. Andrew Clay Shaefer argues on his blog that most of the time we're smart enough to do the right thing with Puppet and use its powers for good not confusion -- except when it doesn't. I still can't make up my mind.
System architectureIn any case, both Puppet and Chef are pretty easy to pick up once you wade through the syntax. They both have their own client/server architecture that compiles these instructions and keeps track of what the machines should have installed. Puppet sticks its instructions in its data stores called PuppetDB and Hiera. Chef uses a mixture of Postgres, Solr, and its own internal structure to keep track of what the client machines should be doing.
These databases end up maintaining a catalog of the state of your vast empire of machines. They track the different files, libraries, executables, configurations, and other details. Then when you push the button, they send reconfiguration lists to the various machines to keep them current.
It's common, for instance, to define a few standard configurations for your enterprise and ask either of these tools to be ready to install them on a new machine. You might have a mail server or a Hadoop node. When a new machine or instance is created, Puppet or Chef will reach out with a list of files that need to be installed.
These lists are often derived from the various plug-ins. Much of the most interesting work is done in these subprojects. Once you get Puppet or Chef running, you'll spend most of your time obsessing about the plug-ins.
Both environments compete heavily with each other, so it's common for them to duplicate each other's innovations. After playing with both, I can't say that one is definitively better than the other. If you want to do things in one, you can usually accomplish the same task with the other or come pretty close.
In general, you can use either tool with most major operating systems. Puppet doesn't come with a Mac installation wizard, but it will work with Mac OS X as it does with most Unixes. Chef is easier to use on the Mac, but still building a full installer for Windows that works with many of the different variants.
While we're on the topic of Macs and Windows, it's worth noting that some companies are using these tools to manage desktop installations in diverse offices. Puppet Labs, the company behind Puppet, notes that Los Alamos National Laboratory uses Puppet to manage its Macs.
Whether you're using the tools for desktops or servers, you'll also be swayed by the status of some of the modules and the state of development. Puppet Enterprise, for instance, offers more support for VMware. If the particular version works well with your cloud stack and the modules are tried and tested, you should choose it.
Puppet Labs vs. OpscodeA harder question is choosing between the business models of the companies. Both fund the development of open source software by selling something else. In Puppet's case, Puppet Labs distributes a rather anemic open source version for free. If you get tired of the command line, you can upgrade to Puppet Enterprise and graduate to a working graphical user interface, better access control, and support. Puppet Enterprise is free for the first 10 nodes, then costs about $99 per node after that. Moderate discounts kick in as your cloud grows into the hundreds and thousands of nodes.
(It's worth noting that Puppet tallies nodes by counting the digital certificates used to encrypt and sign all the commands. The encryption simultaneously protects the entirety of instructions flowing through your empire and gives the company a good way to track how much you're using the software.)
Opscode, the main company shepherding Chef, gives you another choice. Instead of free open source Chef, it will let you install the more capable Private Chef behind your firewall at prices beginning at $120 per month for up to 20 nodes. The company doesn't count individual nodes and makes a slightly ambiguous promise to look the other way if you occasionally use 21 or 22 nodes. It's not concerned about a bit of testing, just sustained use. There aren't many tiers, so crossing a threshold can push up your bill.
If configuring Chef inside your firewall is one more hassle you don't need, Opscode will host the Chef server infrastructure for you for much the same price. It handles making sure there is enough Chef firepower to handle the job of managing your worker bee machines.
Some might find this silly and a potential security hole. To reconfigure your own machines, you send your instructions for your infrastructure to the Hosted Chef machine and it parcels them out to your own machines as needed. But it also makes some sense because keeping Chef running is yet another chore and Opscode is willing to do it for you. It's all just a cloud, and the lines between what's yours and what's theirs is blurring more and more.
Configuring the configurersAs I played around with Puppet and Chef, I started to run into the typical bitrot hassles that plague all computer software systems but especially bedevil large open source ecosystems like Puppet and Chef. Yes, these ecosystems are wonderful wellsprings of creativity, but they're also sources of friction as all that creative genius heads off in different directions. In theory, you can keep plugging in multiple modules and your system will get better and better, but in reality they start competing for different versions of essential modules. The software doesn't plug together as easily as Lego bricks.
For instance, after installing a few modules, my version of Puppet started complaining that it was stuck with version 0.0.6 of the Java_KS library, which it labeled "invalid." The pe_mcollective library wants version 0.0.3 of Java_KS, and it was not going to accept the newfangled 0.0.6, even if that's what all the whippersnappers thought was cool. For grins, I tried the puppet module upgrade command, which sometimes works, but it couldn't downgrade me. It was smart enough to realize that the newest version of the module (1.0.0) wouldn't satisfy the pe_mcollective library's complaints either. I would guesstimate that 90 percent of my time with Puppet and Chef was spent satisfying their configuration gripes.
This is a deep issue for software development that everyone complains about regardless of whether they're running Linux or Microsoft. In a blog post explaining why he switched to a Mac, noted open source developer Miguel de Icaza said of the Mac, "I did not miss having to chase the proper package for my current version of Linux, or beg someone to package something. Binaries just worked."
To be fair, when I installed Chef on my Mac, I had to install plenty of other libraries before it would "just work." A choice aside from the installation guide to Nokogiri, an XML parser, is noteworthy here:
In the world of Puppet and Chef, I often found myself with software that wouldn't "just work" for me, at least until I fiddled a bit more. I had to reinstall modules again and again. It was not uncommon for the software to fail, fail, fail, then work perfectly for inexplicable reasons. To view the glass as half-full, I can say that the software was constantly trying to heal itself and install the right modules. This means it can heal itself a bit in the process of failing, then succeed the next time.
In other words, be prepared to write code or at the very least think like a developer and juggle all of the libraries. The Chef documentation includes a page that explains "just enough Ruby for Chef." If you're working with Puppet, you'll need to get a bit dirty figuring out what to do with the custom language unless you try to reprogram the tasks using Ruby.
Before getting too annoyed by these endless tiny glitches, it's worth remembering the alternative: visiting dozens, hundreds, or perhaps thousands of machines and running or rerunning the same script. Puppet and Chef are leaps and bounds better than writing your own scripts and trying to keep them standardized. They're not a push button tool for managing what is still very much a command-line game that requires command-line code.
Many people will complain about the hassle of interacting with the various plug-in authors in the sometimes sketchy and often hit-or-miss world of open source modules. But working with the open source modules forces the internal users to standardize their scripts and build out an architecture that is often better documented and easier to understand. In other words, using the modules brings discipline.
Yes, you will still have endless internal and external conflicts. Yes, you'll be working in a domain-specific version of Ruby or JSON. Yes, the command line will be a big part of it. But all of this structure will bring some order to the vast empire of cloud machines that you have waiting to be maintained and perhaps even upgraded.
This structure is why people end up both hating and loving tools like Chef and Puppet. They're complicated and sometimes maddeningly unforgiving, but they bring a needed discipline to maintaining all of those servers we bought. We created them with that cloud magic, and now we need some cloud sauce to pour over them to keep that magic alive.
This article, "Puppet or Chef: The configuration management dilemma," was originally published at InfoWorld.com. Follow the latest developments in application development, cloud computing, and open source at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.
Read more about data center in InfoWorld's Data Center Channel.