Mike Brittain

I read this poll on LifeHacker the other day about web-based email vs. desktop email apps.  It reinforced what I believe is the current momentum in web applications these days — that over time, people are going to get more and more comfortable using web apps the way they used to use desktop apps.  It’s been five years since the release of Gmail, which I view as a forerunner in this area, so clearly this isn’t going to be a quick change.

Browser innovations will help users with the perception that web applications are interchangeable with desktop software.  Google Chrome is already working in this direction by reducing the amount of browser UI and allowing the user to focus on sites and apps they are using.  I’m won’t argue that Chrome will be a major browser; it may never be.  I do believe that Google’s intention is to continue swaying the way we look at the Internet. As other browsers follow suit (in some cases), Google’s web applications and sites will all benefit.

As we move forward, many computers, especially in public spaces like libraries and educational computer labs, will use fewer licensed software suites and more subscriptions to web-based applications.  As with Gmail, it doesn’t matter whether I check email on my computer, your computer, a work computer, or a mobile phone — I still have access to the application because modern web browsers provide the baseline of support for these apps.  With stabilization and implementation of features in HTML 5, additional web-based apps will be built and they will continue to look and act like our familiar desktop apps.  Over time, institutions will replace their local file servers, email servers, and parts of their IT staff with outsourced apps that are purchased by subscription and delivered in a browser.

A lightning strike causes EC2 outages and Om Malik blames the “fragility of the cloud,” rather than recognizing that all tech suffers failures.  I’ll say it again, this could have happened to my own servers, or my own data center, and I would have been much further up the creek than if Amazon team was taking care of it.  Besides, one of the most important lessons I have learned from working with AWS is that servers/services should fail, and fail gracefully.  It shouldn’t matter whether that service is “in the cloud” or in your data center.

Grig Gheorghiu wrote up a nice article on handling high traffic sites on EC2.  It’s definitely worth a read for some high-level concepts about multi-tier architectures.  It doesn’t talk deeply on details of EC2 (would have liked to see something mentioned about availability zones for MySQL and load balancers).  One thing I really liked was the concept of using multiple load balancers with round-robin DNS pointing at them.  I’ve been considering this as an option and have played around with HAProxy already.  It’s likely a future step for our new image service at CafeMom.

Cyberduck is a nice Mac FTP/SFTP GUI client that I’ve used in the past for moving files around between my desktop and some web servers.  Turns out they’ve added support for moving your files directly to Amazon S3 and Mosso (RackSpace) Cloud Files.  This means that you can use the same tool that you may previously have used for publishing content to your own web server to instead publish content directly to a self-service CDN.  Amazon uses it’s Cloud Front service to distribute files, and Mosso is supposed to be integrated with LimeLight networks for distributing content from the Cloud Files system.

Just wish I had these services available to me three years ago.  They would have saved me some serious cash on bandwidth commits for CDNs for those silly little projects I was working on.

I just read this short article about Soocial’s hosting architecture which runs on Amazon Web Services.  There was one particular line that echoes what I’ve been saying for a while and I think it is worth repeating:

One of the most interesting things is how the architecture isn’t dramatically different than it would be if you were to build an on-premise version.

In my own experience with hosting on EC2, we built our application on a physical dev server that we already had in place and was running Linux.  It was easy for us (with just a little forethought) to deploy the application on EC2 and S3, and the developers working on the application really needed to know very little about the workings of EC2.

Ylastic is putting a management interface for AWS on the iPhone.  Looks pretty cool.

I am familiar with their name, but don’t have any experience with their product.  I sort of wish these sorts of tools could be open sourced (and there are some) so that I could run the management service on my own servers and not hand over my AWS keys.  Like I said, I don’t have experience with their product, so maybe I’m making an assumption there.

As I’ve said earlier about AWS, it’s an amazing service, but is very much like a raw material.  It’s like having someone hand you the keys to a datacenter, and you don’t even know how to turn on the lights.  Ylastic fits into the category of management vendors for AWS, and I think that Amazon’s ultimate success will depend on management vendors who extend the web services to the layperson.

Matt Spinks put together a Munin plugin for monitoring S3 download speeds, which is now available at Google Code.  I mentioned this in a recent post and wanted to provide an update the the plugin is now published.

I’ve been planning to post some details about download speeds that I’ve seen from S3, and why you shouldn’t necessarily use S3 as a CDN, yet.  Granted, Amazon recently announced that they will be providing a content delivery service in front of S3.  This post has nothing to do with the CDN (CDS).

Scott posted the presentation he gave at the AWS Start-Up Tour.  It’s worth a read, and is a good summary of our business case for building the EC2/S3 hosting platform that I led when I worked at Heavy.

His post includes this graphic of how we measured S3 delivery speeds throughout the day.  Matt Spinks wrote the Munin plugin that generated this graph, and he tells me he’s planning to make that available for others to use.  When it is, I’ll add the link here. It’s now available at Google Code.

As we measured it, S3 is fairly variable in their delivery speeds.  Unfortunately, we didn’t measure latency for initial bits, which would be good to know as well.

My own impression is that it is not a good idea to host video directly from S3 if you run a medium to large web site.  The forthcoming CDN service will probably help with this.  If you’re a small to medium site, you might be happy with hosting video on S3.  Hosting images and other static content (say, CSS and JS files) might also be a good idea if you don’t have a lot of your own server capacity.

For my own use, I’m planning on using S3 to host images and static content for some other sites I run, which use a shared hosting provider for serving PHP.  On a low traffic site, I’d be happy to offload images to S3.  And when the CDN service becomes available, the user experience should be even snappier.

One thing to note if you plan to use S3 for image hosting, look into providing the correct cache-control headers on your objects in S3.  You need to do this when putting content onto S3.  You can’t modify headers on existing content.  More on this in a future post.

I attended the Amazon Web Services “Start-Up Tour” in New York today.  Though I’ve been using AWS for some time now, I learned a few little bits that I thought I would share.  Some of these might have been in press releases that I missed, but I still thought they were interesting.

1. 400K registered developers. Surely not all of these are active developers, or even doing anything large.  But this seems like a pretty good developer base for a set of services that are mostly still in “beta”, and most people consider them to be bleeding edge.

2. “Muck”. This is the term that Amazon uses for all of that infrastructure that you shouldn’t need to build when you’re starting a company… because someone else has already done it.  Stop reinventing the wheel and get focused on your real business priorities.  I’ve heard this term before, but I love it.

3. Start-ups shouldn’t need an ops team. Using AWS, a start-up can get a real infrastructure setup without having to hire an operations team, go through capacity planning, purchase equipment, rent a rack in a colo, deal with power, bandwidth, security, etc.  Companies like RightScale can ease the implementation process, and at least one of the NYC panelists who was speaking at the Start-Up Tour made use of RightScale.  Slightly more expensive for hourly charges, but keep in mind that you won’t need a such a heavy-duty developer to manage your infrastructure.

4. S3 data redundancy. I was aware that data stored in S3 was replicated across multiple nodes, but according to Mike Culver (from AWS): 1.) S3 stripes across multiple nodes, 2.) Rebuilding a node doesn’t reduce performance of the striped system, and …drumroll… 3.) Data is replicated across multiple datacenters.  That’s the good part I didn’t know about.

5. 22 billion served. Well, not really “served”, but as of 2008 Q2, S3 has over 22 billion objects stored.

6. Upcoming products and features. Today, AWS announced that they will be providing a content delivery service for objects stored in S3 — high-speed, low-latency delivery.  Additionally, from what I heard talking with one of the evangelists, AWS is working on a long list of features requested by their customers.  When you think about a high-performance web application, there are a number of moving pieces — front-end servers, app servers, databases, storage, caching, load balancing, DNS, etc.  Lots of “muck” in there that isn’t already provided by AWS, but it sounds like they’re working on a number of these problems.  I’m looking forward to what may come out in the next year.

Amazon Web Services is about to launch what appears to be a globally distributed delivery network for content stored in S3.  Details have just been released today about the new service, which looks to work in conjunction with existing S3 content… which is niiiice.  New domain names are handed out through an API, which presumably will CNAME to a node that is geographically close to the end users POP.

This should continue to heat up the CDN industry, which has already become quite competitive with a number of small players who have come on the scene over the last 2-3 years to challenge the big fish, Akamai.  I’ve had the pleasure of working with Akamai for deliverying Heavy’s content in the past, and can say that they set themselves apart from the rest of the industry by providing a wider range of products than just content delivery.

I’ve used S3 as an origin server to other CDNs in the past, and look forward to seeing how their own delivery network compares in speed, latency, and pricing.  On the surface, this looks amazing for start-ups and other small online businesses who may not be able to afford a large contingent of vendors.  My suspicion is that the pricing will be incredibly competitive, and look forward to seeing actual numbers.