We have Cloud Foundry running in the lab, BOSH deployed, in a vSphere environment.  While I worked with a colleague to deploy that Cloud Foundry, and I focused on understanding what was going on with services deployments, I was perfectly content to have him get us to the point where we could do that Cloud Foundry install.  In other words, he installed BOSH itself.  Recently, however, when he and I were trying to track down an issue we were having with BOSH, I found I really wanted to understand how BOSH is installed.  The short answer is, as reported on many blogs, that it’s just like installing Cloud Foundry – that you install BOSH using BOSH.  But what does that really mean? What are the details?  In this blog I’m aiming to explain WHAT happens when you go through the install steps. You can find those steps here or here, and my goal is not to repeat those instructions, but rather to explain a bit further what’s going on.

The first thing that you do is do some setup on the IaaS, in our case vSphere; networks, folders, etc.  The folders will be used by various installation processes to hold files used during the install. That is, the IaaS itself is used to facilitate the installation, not just host the installed system.  This is a key point that I haven’t found stated elsewhere, but if you watch what is going on in your IaaS (i.e. watching your vSphere console) during a “bosh deploy” you’ll see machines getting created and ultimately deleted – they are in service of the deploy itself.

Okay, so now you have an IaaS environment ready that you can install BOSH onto.  BOSH is itself a distributed application, running on a multitude of machines working in cooperation.  In theory you could create the VMs you need (here it says you’ll need 6) and install the appropriate bits onto each of those machines – put the bosh director on one, the health monitor on another, the message bus on another, and so on. But wait, isn’t that what BOSH is meant to do, provision VMs and install a distributed application onto them?  It is, and that is why the BOSH team has provided something that allows you to install BOSH using BOSH.  The trick then, is how to bootstrap the thing.

Check out the video behind this image - very cute

This is done with two things.  First, the BOSH team has essentially created a virtual machine that has all of BOSH already preinstalled on it. That’s right, BOSH director, the health manager, workers, message bus, etc – all of it on a single VM.  And second, they have given you a tool to install that virtual machine into the IaaS environment that you set up in the first step.  So the next few steps in the instructions have you download the micro-BOSH stemcell (essentially the VM) and install a bosh cli & deployer. The cli has the protocol for interacting with the IaaS built into it, so you can just issue a few cli commands and it does the micro-bosh deployment for you. Some instructions have you create a virtual machine from which you do these steps, but this isn’t strictly necessary; you just need an Ubuntu machine with ruby, git and a few other things on it.  I personally always have Ubuntu VMs I am using for various development activities and are perfectly suited to this set of steps.  So after you’ve installed micro-BOSH following the instructions here or here, you actually have a BOSH up and running.  You can now take “releases” which contain things like packages, jobs and deployment manifests and deploy them with BOSH. And that install was really easy!

But, hang on, the instructions go on – they have me install BOSH.  But, didn’t I just install BOSH? Yes, you just installed (micro) BOSH. And yes, you now need to install (full-blown) BOSH.

Remember that micro-BOSH is deployed on a single VM.  If you are deploying (and monitoring!! – but I’ll talk about that more some other time) a distributed application that isn’t too big or complicated, you could use micro-BOSH and it would work reasonably well.  But ultimately our goal is to deploy more significant applications, like Cloud Foundry, and for that a single VM BOSH just couldn’t cut it.  But the good news is that we now have micro-BOSH to help us install the more sophisticated BOSH deployment.

One of the components in that micro-BOSH deployment is something called the BOSH director, which, as the name implies, orchestrates all of the things that BOSH does for you. So to deploy anything with BOSH you basically tell the BOSH director what you want to do, where it can find the bits it needs for the job, and what environment you want to install those bits into. Let’s take those things in reverse.

Remember that BOSH will stand up VMs onto which it then installs things.  What does that VM look like?  This is where stemcells come in again.  The instructions have you download, and then upload via micro-BOSH, the bosh stemcell.  The BOSH stemcell is different from the micro-BOSH stemcell in that it contains only a base operating system plus a BOSH agent.  Recall that the micro-BOSH stemcell has all of BOSH on it.  I find it a bit ironic that the micro-BOSH stemcell is actually much “bigger” than the (full) BOSH stemcell.  Really! Have a look at the file sizes for the two stemcells you’ve downloaded.

The bits that are needed for the application installation are bundled in a release.  You find the BOSH release in the bosh repository on git.  The structure of a BOSH release has been covered in numerous blogs and videos, and I won’t cover the details here. The instructions will have you git clone this repository to get all the bits you need; you’ll have to modify a few settings for your vSphere environment.

Finally, you need to tell the BOSH director what you want to do, and this is accomplished with a series of bosh cli commands, as covered in the installation documents.  What is important to note in the instructions is that when you are installing BOSH (using micro-BOSH) you target the BOSH director that is a part of micro-BOSH.  Once the (full) BOSH install is done you need to change which BOSH director you are targeting.  If you don’t and you try to do something like a Cloud Foundry deployment you’ll be attempting that with the micro-BOSH and that is not likely to go well.