Monthly Archives: March 2014

Vagrant and Puppet

In the last article “Automatic Provisioning“, hopefully I’ve convinced you to look into automatic configuration management and why it’s important. There’s a bit of a learning curve to Vagrant and Puppet, but I promise it will save you time and headaches in the long run.

I’m just going to point you to some resources online. I don’t expect you to read all of it, but read enough to get a sense of them. At the very least, you have a bookmark for where to look when you run into an issue or need to understand my example better in the follow-up article.

Before you start reading, here’s a image to describe how the pieces work together

vagrant

[source]

The picture above illustrates the fact that you start with a box, like Ubuntu (which can be downloaded from places like vagrantcloud.com), apply Vagrant and a provisioner, like Puppet, and then get a Ubuntu VM set up exactly as you need it.

A word about Vagrant

Vagrant provides you with hooks to a set of tools to reproducibly configure work environments. It will help you create and manage a virtual machine, so you can isolate different work environments on your development machine. It will also leverage different “provisioners” to automatically set up the virtual machine exactly as you need to (more about this later).

You’ll want to get started by looking at the Vagrant’s Getting Started Guide.

Once you’re done, you should get an idea that vagrant takes in a virtual machine box, provisions it, and outputs a work environment. The Vagrantfile is the config file that manages all of it.

Here’s another resource that was immensely helpful: VagrantCloud
This site helps you find the virtual machine boxes you need to start off with. Are you looking for an Ubuntu 12.10 server? How about a Debian server with Puppet installed? It’s all here. You can use and share your vagrant boxes with others.

A word about Puppet

Somewhere along the way of reading the guide above, you should have encountered something called Puppet. This is a provisioner. It configures your system. There are others, notably Chef, Docker, Salt, and even basic shell scripts.

I chose Puppet because it seemed to be the most mature with the largest community (this is important and I’ll explain why later). Puppet uses json which is pretty simple to read and write. It’s a declarative language so instead of focusing on “how” to install something, you can focus on “what” to install.

Vagrant lets you mix and match provisioners, so you’ll see that I’ll throw in some shell scripts when it’s appropriate and easier. But for the most part, if you keep all your configuration to Puppet, it offers you a standard interface to the build of the system.

Get started with Puppet here:

After you learn the syntax of the puppet language, you might wonder if someone else has already tried to install and set up what you want . Click here to have your mind blown

This where the community aspect comes into play. There’s a large community of Puppet users who have packaged their installation scripts into Modules (you should have read about what Modules are in the Puppet tutorial). And not only that, they’ve exposed the various settings of the component you want to install.

What this means is that, not only can you install mysql, but you can also set up users, grant privileges, create databases, all before a developer begins working in that environment. This is the magic of automatic configuration management tools.

In the next article, I will detail how I used Vagrant and Puppet to build a complicated system that took days in its original build, but just minutes to run in its current form.

Tagged ,

Automatic Provisioning

If you’re like my small company, we do not have a dedicated IT department. Every developer manages their own development environment. As you can imagine, the systems that we build get out of sync very easily. (…I’m on java7, why are you using java6? I’m using apache, you’re using tomcat? Did you install mysql?…)

Even worse, is when it comes to the discrepancy in the production environment and development environment. The production environment may have evolved over time, with new components added and old components upgraded. But how were they installed–using a zipped binary, from a source tar, or the distro package manager? These become relevant questions when you want to remove, upgrade, and just look for files.

A lot of companies have resorted to taking snapshots of their production server over time. This is what’s referred to as the Golden Image. This can be expensive. It also isn’t an elegant solution and will bring headaches when you need to remove, upgrade or add new system components.

So to summarize, here’s a list of problems off the top of my head (there are probably more):

  • Developers are not IT people

    • Every developer has a different development environment, with different versions of various components (java6 vs java7, tomcat6 vs tomcat7,…)

  • Developers work on multiple projects

    • How do you resolve the discrepancy if one project needs tomcat6 and another needs tomcat7?

  • Environments evolve and are hard to maintain

    • You upgrade to java7, but did you really remove all references to java6?

  • Development environments are not aligned with Production environments

  • Production environments need to be reproducible

    • Current using the “Golden Image” solution which has flaws

I believe the right way of doing this is using automatic configuration management tools like Vagrant along with Puppet.

Configuration Management relies on the art of automation–you start with a generic machine, automatically install and set up the system to get it to a development- or production-ready state.

Here is an excellent article of the two approaches.

In the next article, we’ll take a look at Vagrant and Puppet a little more closely.

Tagged , , , ,