Matt Spinks put together a Munin plugin for monitoring S3 download speeds, which is now available at Google Code. I mentioned this in a recent post and wanted to provide an update the the plugin is now published.
Monthly Archives: September 2008
Cloud Computing / Comments Off on Munin Plugin for Testing S3 Speed
Cloud Computing / Comments Off on Download Speeds from Amazon S3
I’ve been planning to post some details about download speeds that I’ve seen from S3, and why you shouldn’t necessarily use S3 as a CDN, yet. Granted, Amazon recently announced that they will be providing a content delivery service in front of S3. This post has nothing to do with the CDN (CDS).
Scott posted the presentation he gave at the AWS Start-Up Tour. It’s worth a read, and is a good summary of our business case for building the EC2/S3 hosting platform that I led when I worked at Heavy.
His post includes this graphic of how we measured S3 delivery speeds throughout the day. Matt Spinks wrote the Munin plugin that generated this graph, and he tells me he’s planning to make that available for others to use. When it is, I’ll add the link here. It’s now available at Google Code.
As we measured it, S3 is fairly variable in their delivery speeds. Unfortunately, we didn’t measure latency for initial bits, which would be good to know as well.
My own impression is that it is not a good idea to host video directly from S3 if you run a medium to large web site. The forthcoming CDN service will probably help with this. If you’re a small to medium site, you might be happy with hosting video on S3. Hosting images and other static content (say, CSS and JS files) might also be a good idea if you don’t have a lot of your own server capacity.
For my own use, I’m planning on using S3 to host images and static content for some other sites I run, which use a shared hosting provider for serving PHP. On a low traffic site, I’d be happy to offload images to S3. And when the CDN service becomes available, the user experience should be even snappier.
One thing to note if you plan to use S3 for image hosting, look into providing the correct cache-control headers on your objects in S3. You need to do this when putting content onto S3. You can’t modify headers on existing content. More on this in a future post.
Mobile / Comments Off on Future of Mobile (from Andy Rubin)
it’s safe to say that the mobile phone may be the most prolific consumer product ever invented.
In comparison to computers that we may have owned…
The phone that you have in your pocket, pack, or handbag is probably ten times more powerful than the PC you had on your desk only 8 or 9 years ago.
I was in real awe when I got my iPhone a year ago (nearly to the day!) and remember tinkering with it on the subway. At first I thought, “this is the most amazing item I’ve ever owned.” I think that still holds true. Nothing else does quite as much, in as small a case.
I also remember saying to a number of people, “if this is the phone I have today, what kind of phone will my kids have 15-20 years from now?”
Rubin describes some of his predictions. Read on…
Cloud Computing / Comments Off on Notes from AWS Start-Up Tour (NYC)
I attended the Amazon Web Services “Start-Up Tour” in New York today. Though I’ve been using AWS for some time now, I learned a few little bits that I thought I would share. Some of these might have been in press releases that I missed, but I still thought they were interesting.
1. 400K registered developers. Surely not all of these are active developers, or even doing anything large. But this seems like a pretty good developer base for a set of services that are mostly still in “beta”, and most people consider them to be bleeding edge.
2. “Muck”. This is the term that Amazon uses for all of that infrastructure that you shouldn’t need to build when you’re starting a company… because someone else has already done it. Stop reinventing the wheel and get focused on your real business priorities. I’ve heard this term before, but I love it.
3. Start-ups shouldn’t need an ops team. Using AWS, a start-up can get a real infrastructure setup without having to hire an operations team, go through capacity planning, purchase equipment, rent a rack in a colo, deal with power, bandwidth, security, etc. Companies like RightScale can ease the implementation process, and at least one of the NYC panelists who was speaking at the Start-Up Tour made use of RightScale. Slightly more expensive for hourly charges, but keep in mind that you won’t need a such a heavy-duty developer to manage your infrastructure.
4. S3 data redundancy. I was aware that data stored in S3 was replicated across multiple nodes, but according to Mike Culver (from AWS): 1.) S3 stripes across multiple nodes, 2.) Rebuilding a node doesn’t reduce performance of the striped system, and …drumroll… 3.) Data is replicated across multiple datacenters. That’s the good part I didn’t know about.
5. 22 billion served. Well, not really “served”, but as of 2008 Q2, S3 has over 22 billion objects stored.
6. Upcoming products and features. Today, AWS announced that they will be providing a content delivery service for objects stored in S3 — high-speed, low-latency delivery. Additionally, from what I heard talking with one of the evangelists, AWS is working on a long list of features requested by their customers. When you think about a high-performance web application, there are a number of moving pieces — front-end servers, app servers, databases, storage, caching, load balancing, DNS, etc. Lots of “muck” in there that isn’t already provided by AWS, but it sounds like they’re working on a number of these problems. I’m looking forward to what may come out in the next year.
Food and Cooking / Comments Off on Random Thoughts from My Kitchen
I just spent the last 3 hours screwing around in my kitchen with a variety of things, dinner being one of them. Here are some random thoughts and learnings…
- Broccoli rabe is a pretty good alternative as a dark green vegetable to the spinach, broccoli, and broccolini that I usually eat. Given the tight quarters in my kitchen, it’s a good idea to boil this ahead of time (about 6 mins), shock, and store until ready for dinner. About 5-10 minutes in a saute pan to finish it. Fresh Direct sent a pretty good batch of it.
- I really should eat more bluefish.
- There are a number of decent videos on how to render clarified butter, but nobody warned me that when the butter is first melting that it might sit quietly for a while, as it plots to suddenly explode all over my stove and floor. I barely escaped with my life.
- Also, nobody warned me how good it would smell when it was finished… it was heavenly.
- Baby octopus is incredibly tender when it is boiled for about 45 minutes. It is also one of the scariest looking items I’ve ever cooked. It was like cooking an alien. I still need to figure out just what to mix it with, though. Doesn’t have too much flavor on its own.
Cloud Computing / Comments Off on Amazon Is Launching a CDN
Amazon Web Services is about to launch what appears to be a globally distributed delivery network for content stored in S3. Details have just been released today about the new service, which looks to work in conjunction with existing S3 content… which is niiiice. New domain names are handed out through an API, which presumably will CNAME to a node that is geographically close to the end users POP.
This should continue to heat up the CDN industry, which has already become quite competitive with a number of small players who have come on the scene over the last 2-3 years to challenge the big fish, Akamai. I’ve had the pleasure of working with Akamai for deliverying Heavy’s content in the past, and can say that they set themselves apart from the rest of the industry by providing a wider range of products than just content delivery.
I’ve used S3 as an origin server to other CDNs in the past, and look forward to seeing how their own delivery network compares in speed, latency, and pricing. On the surface, this looks amazing for start-ups and other small online businesses who may not be able to afford a large contingent of vendors. My suspicion is that the pricing will be incredibly competitive, and look forward to seeing actual numbers.
Finding a small issue with Google Analytics’ tracking code when served on pages with the application/xhtml+xml mime-type… It doesn’t work.
This probably affects only the “latest” tracking code, ga.js, and not the legacy code, urchin.js.
The fix is actually pretty easy, despite the fact that I couldn’t find any official documentation about it from Google. Take a look at the legacy code for inserting urchin.js on your page. It uses a normal script tag, instead of using document.write() to insert the tag — which is legal.
Now look at the JS code used to insert ga.js. Assuming you’re not serving an SSL site, you can replace the document.write() with a static script tag. Something like the one below:
If you are using SSL on your site, along with the application/xhtml+xml mime-type, then it’s up to you to deal with switching the hostname and protocol in the URL in the appropriate pages.
You should not make this change to the tracking code unless you need to (i.e. you’re serving XHTML and tracking pixels are not firing properly).
Mobile / Comments Off on Polishing Mobile Sites with Validation Tools
I’ve been cleaning up some details on the mobile interface for One tsp., specifically so that I can serve the site properly using application/xhtml+xml to browsers that can handle that sort of thing. Unfortunately, that means that when things go wrong, they really break.
Mind you, the site seems to be working just fine as text/html in most browsers and emulators I’ve checked. I’m using this more as an exercise to make sure that browsers I don’t know of, or are more strict on markup, can display the site in a meaningful manner. I also want to avoid having the site mangled by a mobile proxy or transcoder, and I suspect that serving proper mobile pages will allow me to escape them.
To improve the site, I’ve been using the ready.mobi checker and the W3C mobileOK checker. These tools are called “checkers” and not “validators” because they really strive to do more than just validate your markup. They look for valid CSS and compliance with mobile recommendations and best practices (proper use of CSS measurements, HTTP caching headers, links to non-mobile content, etc.).
Both tools are worth checking out if you’re building a mobile site.
Any other tools you use that are worth a recommendations?
Google, WWW / Comments Off on Making Application Shortcuts for Chrome
Google has a page listing some details for webmasters about their new web browser. My favorite part is in regards to handling application shortcuts which I figured out from Gmail’s site. It’s a simple addition of “application-name” and “application-url” meta tags, along with a meta description and link to an icon file, which you probably already have on your site.
Application shortcuts are probably my favorite feature of Chrome, and the reason why I think that it is an important browser. These shortcuts will help users and businesses to change their mindset about software-as-a-service applications. They make web applications look like real software. We used to have a desktop icon for Eudora or Outlook Express. Now we have one for Gmail. In the future, I suspect we’ll see more people replacing their installations of Microsoft Office with a shortcut to Google Docs, Zoho, ThinkFree, or Live Workspace.
WWW / Comments Off on What I think about Google Chrome
I didn’t feel like posting an entry about Chrome, but I’ve had a lot of opinions about why I think it’s good. Will I use it as? That’s a good question. We’ll need a Mac version, first, but I think I’d like to make the transition.