Running a Local (as in, on your laptop) Cloud Foundry Instance
More than two years ago on the CF blog the availability of Micro Cloud Foundry was announced – this was a version of Cloud Foundry that would run on a laptop. It came in the form of a vmdk that you could run either using VMWare Workstation or VMWare Fusion. Later posts talked about a release lifecyle that would provide frequent updates to the Micro release, so as to keep it in synch with the constantly-being-updated public offering running on cloudfoundry.com. It was an interesting experiment that had legs for more than a year, but didn’t end up panning out all that well; I can share a personal anecdote that illustrates one of the reasons. More than a year ago when I was with EMC, I was working with some product groups that wanted to use CF, but not via cloudfoundry.com, but rather with a customized version of CF. They deployed a private cloud in their data center but then wanted to enable their developers with a matching “micro” instance running on their personal workstations. We looked into the process of getting a customized version of Micro CF but in the end the answer was convoluted enough to keep us from making any progress. As it happens, we weren’t alone.
And now, here we are on the verge of our first supported release and I’m happy to report two way cool things. First, there are now numerous, very viable options for getting a local CF instance up and running, and second, the CF community has become so vibrant that two of the three options I’ll describe have come from outside of Pivotal. In this post I’ll share with you my personal experiences with and opinions on these three options.
I’ll go through these in the order in which they appeared on the scene, which is, incidentally, the order in which I began using them.
CF Vagrant Installer from Altoros
In April of this year the good folks at Altoros released the cf-vagrant-installer. This project required you have vagrant and a hypervisor (no license fees if you use Virtual Box), as well as ruby installed on your host machine (laptop), and then allowed you to clone the git repo and run the “vagrant up” command. Worked really well if you timed it just right, but because the standing up of the CF components was done via chef scripts that are a part of the project, keeping the installer up to date with the CF releases has proven challenging.
This was the first of the three options that I used earlier this year and I was incredibly grateful to have it. That said, because of the aforementioned challenges in keeping this in synch with the evolving CF releases, and dwindling activity in the project, I no longer recommend it.
CF Nise Installer from NTT Labs
Just a month after the release from Altoros, NTT Labs released the cf_nise_installer. The install can be run with the help of vagrant, or you can create an Ubuntu 10.04 machine yourself (virtual or physical) and run two commands WITHIN that VM. You don’t need to install ruby on your host (really nice if your laptop is a Windows box) – in fact, if you run the installer in your own Ubuntu machine then there are absolutely NO dependencies on software running natively on your laptop. The basic mechanics of how the Nise installer does its thing is that it leverages a BOSH emulator, also created by NTT Labs, and the install then 1) clones the cf-release repository (which is the official release vehicle for the OSS Cloud Foundry), 2) creates a release using the BOSH cli and 3) deploys that release onto the VM via this BOSH emulator, called Nise BOSH. Because the deployment leverages cf-release directly, it doesn’t get out of synch with the CF release – that is, it IS the CF release. By default the cf_nise_installer will install the latest “final build” as found in the releases directory. It’s true that the BOSH emulator has to keep up with changes to BOSH and the deployment manifest must also be updated, but yudai is remarkably responsive and it’s a rare day where someone reports an issue on one of the mailing lists and doesn’t have a resolution from him in just a couple of hours (I’m honestly not sure that the guy ever sleeps ;-)).
In spite of the third tool I am just about to describe, the cf_nise_installer remains the option I most often recommend for standing up a local cloud foundry quickly, because, well, it just works – and in about 30 minutes.
BOSH-lite
While the commit history dates back to June of this year, it was more like August when bosh-lite matured enough for consumption. This project was initiated from within Pivotal and, most simply put, it is an implementation of a Warden CPI for BOSH. If your not thinking this, let me seed this in your consciousness…
A warden CPI. Oh, now, That. Is. Cool!
BOSH is a product that allows you to deploy and manage complex distributed applications, like the Cloud Foundry Runtime and Services, over the top of any IaaS. BOSH just needs to be able to provision virtual machines and then it lays things down onto those machines and monitors them too. We’ve had support for VMWare, Openstack (provided in partnership with Piston Cloud) and AWS for some time, and NTT recently announced one for CloudStack (and there are more in the works too). Warden is a system that serves up light-weight Linux containers – that is, super-lean virtual machines. The CF runtime provides application isolation by hosting deployed applications inside of warden containers, and now bosh-lite uses that same technology for the VMs into which CF components are deployed.
With a Warden CPI for BOSH, you now have the ability to run full BOSH on your laptop, and hence you can do a full BOSH deployment of CF on your laptop. I know I said it already but, whoa, that is way cool.
That said, there are a few gotchas. First, if you clone the repo today you will find that by default, the vagrant config uses 6GB of RAM – pretty hefty (I have been able to do it with 3 or 4 Gig). You will also need to install a number of things on your laptop to be able to do the install. All told, I’ll confess that it took me the better part of a day to sort through all of the dependencies, clean up my laptop sufficiently and get it all up and running. And I recently witnessed a partner have roughly the same experience.
One of the best things, however, is that once you have bosh-lite installed, then the deployment of CF proceeds in the same manner as it would if you were deploying CF into the cloud. This has the obvious advantage of allowing you to gain necessary experience with an important toolset, BOSH, but that is, admittedly, a fairly significant burden for the newbie. Another advantage that the bosh-lite approach has over CF Nise is that because the CF components are isolated from one another in the Warden containers, you won’t have issues with various CF pieces bumping into each other – CF Nise cannot, for example, deploy the CF runtime and most of the services on the same box.
To summarize:
- I no longer recommend the vagrant installer from Altoros.
- If you are looking to get the simplest CF instance stood up on your laptop, don’t need CF services right away (but you might be working on creating your own service broker – that’s okay), then I highly recommend cf_nise_installer.
- If you have enough RAM and are looking for a full-blown deployment of CF with services, and/or you are primarily interested in learning BOSH, then go with bosh-lite.
I have options 2 and 3 operational on my laptop and depending on the task before me I choose one over the other.
Manuel Garcia
As always, your posts are just great.
Thank you very much for you compliments about Altoros Vagrant installer.
Back in the days, where no local CF VM were available, it was an awesome tool to get CF not only up and running to push and manage apps but also to get it in such a way where you can browse the code, play with configurations and debug.
It really helped us to learn CF internals.
Right now it hasn’t been updated with latest versions of each component and, as you say, it is hard to keep up against cf-releases versions.
I agree with you in your suggestions.
Lately, thanks that I have enough RAM, I am using bosh-lite with very good results.
Thanks for the post.
Manuel
Rohit Kelapure
We have found bosh-lite difficult to use in practice with openstack and other IAASs, It looks ubuntu development environment with virtualbox is the way to use bosh-lite
Matthew Sykes
So, bosh-lite is intended to deliver a Linux container based environment for experimenting with bosh deployable systems; it’s not intended to be a production deployment. From my perspective, it’s an amazing productivity enhancement for deploying and testing cloud foundry and/or services.
When it comes to a single vm openstack deployment of bosh-lite, if I need an all-in-one vm deployment, I’ve had a lot of luck. While it’s not nearly as quick to bring up (with the vagrant openstack plugin) it works well and can be captured with a snapshot for actual deployments. The only “trick” is the iptables config required to access apps and the use of a (bosh supported) gateway when remotely accessing the ‘vms’ hosting the jobs.
In the end, I generally agree with Cornelia’s evaluation. I will also say that while the Alteros project is a bit out of date, with subtle changes, it’s really useful for build pack testing. Basically, you have to pick the right tool for the job. For me, in 90% of my cloud foundry work, that tool is bosh-lite running on my laptop.
Andy Bravery
Would be interested to know if anyone has ever got the bosh-lite approach to work on Windows?
I tried numerous times, with and without cygwin, and was not able to get all the dependancies in place to get to the end.