Cloud Magic and a Talking Monkey

Posted in: Blog, Technology on October 26, 2009

Security guards disallowed us from entering our data center because of an invoice dispute, which resulted in a geek-strewn, network-cable-tied, flying-hard-drive embroiled bloodbath. I only survived because I happened to have my trusty flask of high-potency 151 rum with me, which I used to cauterize my wounds and calm my nerves.

That first part is something that you won’t likely hear me say in the next few years, with any luck. We won’t enter our data center, partly because we don’t know where it is. I’ve never physically seen or touched our servers. We invested millions in physical hardware at my previous company, so the notion of building out a highly-scalable system in the cloud is both liberating and a bit disconcerting (I like pushing buttons and blinking lights). But it’s all made possible through the magic of server virtualization and our friends at Amazon AWS.

Of course, there are costs and benefits that accompany this kind of technical voodoo, both financial and practical. In the process of considering the costs and benefits of going with AWS vs. other virtual hosting providers (like Slicehost – they rock, and we use them as well) and vs a traditional co-located model, we considered many things: fixed costs, headcount/expertise requirements, flexibility, etc. It was a very scientific process, involving a gigantic spreadsheet, an obscure algorithm, a jar of pickles and a talking monkey…  I don’t have time to explain the nuances, but will summarize our findings for you. As it turns out, flexibility wasn’t critical; I can barely touch my toes but was still able to boot up an EC2 instance without a problem. Expertise requirements: the guy that set up our servers could be best described as a talking monkey; make of that what you will. But one of the nice things about AWS is that you only pay for what you use, at a very granular level.

For example, if I wanted to do some load testing, within a few minutes and clicks I can instantiate n of our custom-built web servers, use them for an hour, shut them off, and get charged an hourly rate. That rate will vary depending on bandwidth and disk I/O, amongst other things. Very cool. Maybe the cable companies will follow suit. I’d keep my money on Amazon.

That kind of “just in time” availability complements our product development approach and strategy very well: on publisher request, we’ll turn on a dime and build new product features on the fly. If you are going to jump through it, your back end has to be flexible, or you end up looking like one of those dudes from the Bodies exhibit.

Jeff