There are a lot of things we want to do with Cloud Foundry, at the moment we are focusing on adding services such as Cassandra.  We’ve been studying the code for existing Cloud Foundry services (i.e. Postgres) and we’ve deployed the sample Echo service, a couple of times.  This was a pretty significant investment that is now set to pay off as we really feel like we know how to tackle the problem.  What I’m going to do in a series of posts is first (in this post) update the instructions you can find here on deploying the Echo service to a non-BOSH based deployment of Cloud Foundry, second more fully explain what the Echo service and client are doing and clarify what the various components are, and third, detail what we did to get things running in a BOSH-based Cloud Foundry deployment.  This mirrors the progression of our investigative work and I hope you might find it useful.

There are two versions of a guide on deploying the Echo service (and a client app utilizing it) – one on the support site and another in github. They differ slightly but both are about the same vintage (September 2011) and a bit out of date with the current Cloud Foundry code.  Also, I had to make some changes to the Echo service itself to get things running.  Here are the details – I’ll generally work from the version that is on the support site:

Create Single Node Cloud Foundry Instance

First things first, I did this deployment of Echo to a single-node Cloud Foundry instance that I installed using these instructions (as a services developer, micro-cloud foundry is NOT the local cloud solution).  I started with the Ubuntu 10.04 desktop image simply because I have been spending a fair bit of time poking around the contents of Cloud Foundry files and doing so with multiple windows is nice; you could also use the server version . The Cloud Foundry install went pretty smoothly, though I did have to restart it once because one of the required resources wasn’t accessible (reported “Could not reach http://rubygems.org/”) the first time through; that is,  after  seeing an error, I simply executed the following command again and things continued to a successful install.

bash < <(curl -s -k -B https://raw.github.com/cloudfoundry/vcap/master/dev_setup/bin/vcap_dev_setup)

Do a quick check to make sure it’s all installed and working properly – start the server (step 3 in the instructions) and push an app – something like this one is super simple.

Make a snapshot of your image now.

Adding Echo

I’ll go into more details in the second part of this blog series, but there are three pieces to the Echo service; the echo gateway, which services provisioning requests (i.e. as a part of a push), the echo node, which does the actual work of provisioning the service, and the service itself.  All three are covered in the original doc, but it’s not entirely clear what is happening where – I’ll try to crisp that up a bit here.

Echo Node and Echo Gateway

In the original doc, this is what is happening in steps 1-8 – here are my updates to those steps:

Step 1: Do just what they say. One thing to note, however, is that when you are adding your own service, not echo, to a non-BOSH based cloud this will not be enough.  There is an innocuous little piece of the cloud foundry code base (I would call this a bug, but it’s an engineered bug) that has service names hard coded in it.  At https://github.com/cloudfoundry/vcap/blob/master/dev_setup/lib/vcap_components.rb#L399 (and a few more lines below it) you’ll see that there is a list of services – and you’ll note that echo is already in that list; that is, echo is already sort-of included in the base cloud foundry vcap code base.  If you are starting with something net-new, this is one place you will also have to make changes.  I’ll try to get to a separate post on this little gem, but in short, the names included in this list are used in the directory paths to the node and gateway bits of the service – that is how start and stop scripts are found, for example.

The good news is that this engineered bug is gone in BOSH-based deployments of cloud foundry.  Makes sense, BOSH is concerned with the components of the deployment, cloud foundry itself no longer is.  More on that in part III.

Steps 2 and 3: Do them just as described, though the lines for step 3 are already present in that file when cloud foundry is installed.

Step 4 is no longer needed.  In fact, the bin/services directory no longer exists in the current version of cloud foundry vcap.  These files were the executables for the service gateway and service node and were just proxies for the files that held the real startup details.  It looks like the startup mechanism changed from earlier versions.  Now, the aforementioned vcap_components.rb file has encoded into it THE place that the executables for the node and gateway are to be placed – convention over configuration!  That convention is as follows: Given the <name> of a service the node and gateway executables are placed as follows:

.../cloudfoundry/vcap/services/<name>/bin/<name>_node
.../cloudfoundry/vcap/services/<name>/bin/<name>_gateway

Step 5: This step involves two things 1) dropping the node and gateway implementations in the right location and 2) configuring them appropriately.

As I already mentioned, the echo service is mostly already included in the cloud foundry vcap code base so the first part already done for you.  I took a close look at the implementation that is in the vcap repository and compared it to that which is in the echo_sp.zip file linked from the original article, and it looks to me like the code that is part of vcap is more up to date.

For configuration, you are first instructed to copy the echo_node.yml and echo_gateway.yml files from the …/cloudfoundry/vcap/services/echo/config directory to the …/cloudfoundry/.deployments/devbox/config directory.  While this step isn’t absolutely essential because on startup if the .yml file isn’t found in the latter location it will search the former, making the copy begins to give you an idea of how a BOSH deployment handles this – think of the copies under vcap… as a template that is instantiated into the .deployments… location; more on this is part III.

When you are editing the yml files in the .deployment… directory, the instructions are quite out of date – there are more required properties for those components than there were a bit more than a year ago; even the .yml files for echo that are part of the vcap source (which is what we are using) are not up to date.  The echo_gateway.yml should read:

---
cloud_controller_uri: api.vcap.me
service:
  name: echo
  version: "1.0"
  description: 'Echo service'
  plans: ['free']
  default_plan: free
  tags: ['echo', 'echo-1.0', 'echobased', 'demo']
  timeout: 15
  supported_versions: ['1.0']
  version_aliases:
    "current" : "1.0"
ip_route: localhost
index: 0
token: changeechotoken
logging:
  level: debug
mbus: nats://nats:nats@<nats_host>:<nats_port>/
pid: /var/vcap/sys/run/echo_service.pid
node_timeout: 2

And the echo_node.yml should read:

---
plan: free
capacity: 100
local_db: sqlite3:/var/vcap/services/echo/echo_node.db
mbus: nats://nats:nats@:<nats_host>:<nats_port>/
base_dir: /var/vcap/services/echo/
index: 0
logging:
  level: debug
pid: /var/vcap/sys/run/echo_node.pid
node_id: echo_node_1
port: 5002
host: <host_ip_address>
supported_versions: ["1.0"]
default_version: "1.0"

Note the comment in the original instructions that indicates a preference for IP addresses.  Hint – have a look at one of the other service _node.yml or _gateway.yml files and grab the mbus value from there.

Steps 6 & 7: Exactly as written in the original

A this point you have the echo_node and echo_gateway running, BUT the actual echo service is not yet running.

Echo Service

The echo service is a very simple java program that simply starts listening over sockets on the port you supply as a command line argument.  In the original instructions this was the very last step (I thought it a bit odd that the instructions had you push an app before the service was available – it’s okay, but a bit odd) – that is, step 3 in the section on “Consuming the echo service” (again, a misleading title as only steps 1 & 2 are part of the consumption; step 3 provides the service).

I wasn’t able to get the ssh tunneling (step 4 in the cloud foundry vcap installation instructions) working on my machine, instead I put entries for *.vcap.me in my dev VM hosts file, so I had to modify the Echo service to open the socket on my IP address of 192.168.1.xxx instead of on 127.0.0.1.  More details to follow in part II of this series.

But then I started the service with:

java –jar echo_service.jar –port 5002

Echo App

Okay, so now we are ready to deploy an application that consumes this service.  The original instructions have you do it in two steps, provisioning a service and then binding it to an application during the push.  You can do all of this during the app push.  The app.war posted with the original article works fine, provided you have everything configured properly; if not, you’ll probably get a stack trace (again, I’ll cover this in the second part of this series).  A couple of things to be careful of while you are doing the push.

1. Choose java, not java7 for the java version – the app is complied with 1.6.
2. When you provision the echo service, make sure to use the name myecho for the name of the instantiated service.

And that should do it – browse to echotest.vcap.me and give it a shot.