extending the cloudstack api

Like I mentioned in my last post, I’ve been toying with a CloudStack based private cloud. Let me try to paint a picture of what we’re trying to accomplish with cloudstack:

1. Our dispatch software decides it needs a box of flavor ‘foo’ and calls out to the API to deploy a new box
2. Dispatch software waits until box is up before dispatching a job to the box
3. Dispatch software monitors running job. When job is done, terminates the VM instance

Nothing revolutionary there. The part I was having trouble with was in step 2: notifying the dispatch software in some fashion that the box was ready to have a job dispatched to it. Out of the box, cloudstack supports a machine state called ‘Running’, but that wasn’t good enough for us as ‘Running’ simply meant that the VM had been started…it doesn’t mean that it’s responding to ping or has had configuration management (in our case puppet) run on it.

So, I wanted a way for puppet to notify the dispatcher that the box was fully configured and ready to accept a job from the dispatcher. Let’s call this the ready state.

There were a couple of ways I came up with to accomplish this:

• Have the dispatcher include a callback url in the userdata when creating the VM
• Use some sort of flag in the userdata

I ultimately decided to implement a combination of the two because I didn’t want to impose any further requirements on the team building the dispatcher and I wanted to implement some other additional functionality that could only be done by adding additional commands to the API. Unfortunately, via the CloudStack API, I couldn’t find a way of reading the userdata associated with a VM. I could set it (via the deployVirtualMachine call) and update it (via the updateVirtualMachine call), but I couldn’t get it.

I decided to spin up my own api extension to add a getUserData call. You can find the source code for my api extension on Github cloudstack-api-extension.

I added an extra run-stage to my puppet runs that runs after all other run stages. What it does is determine if the machine is a cloudstack VM and if so, it calls back to a php script on the cloud manager passing its instance ID (which it knows by wget’ing against the virtual router) which then makes a call to getUserData and then calls updateVirtualMachine to set the userdata = userdata + ready=1.

Kind of a convoluted mess, but now the dispatch manager knows for certain not only when the box is up, but has finished configuration management.

I mentioned that I also wanted to add some other commands to the API…I wanted to add the concept of what I’m calling a bundle to CloudStack. The people running the dispatcher don’t really care what foo is, they just know it as foo and they know when they need another foo. They don’t really care if foo is a CentOS 6.0 or a 5.7 box, has 2 cores or 4, etc., they just need another foo. So I wanted to bundle all of what makes a foo into a foo into a bundle, so when they need it, they just say, give me a foo and we give them whatever corresponds to a foo.

So a bundle is just a named collection of a zone id, service offering id, disk offering, userdata, and a template id. I added two commands to the api: listBundles and deployBundle

One nice thing about adding bundles is that it allows me to update what comprises a bundle without having to update the dispatcher/dispatcher configuration.