Last weekend Etsy put on a mini tech conference outside of SxSW, Moving Fast at Scale. Etsy has been growing fast. When I started last February, we had about 20 engineers. Today we have somewhere around 65. A fundamental goal for us is to keep the process of shipping code simple, as if we only had two or three people sitting around a single table working on the site. Rather than building massively complex rules around communication of changes, review processes, and release management, we’ve created tight feedback loops using continuous integration and rich application monitoring. I spoke on the latter at SxSW.
We collect an enormous number of graphs and metrics around our servers and application code. The number of graphs we generate from our application code is over 16,000. How do we collect so many metrics? We keep the process super simple.
Logster is the tool we use for parsing logs every minute and aggregating interesting bits that we can shoot off to either Ganglia or Graphite. We log a lot of events from our application code, but it’s really hard to see trends in those logs as they’re shooting across your screen at hundreds of lines per second. That’s where Logster comes in.
StatsD is a network daemon that listens for metrics over UDP, then aggregates similar metrics and sends them over to Graphite. The client code we use for StatsD at Etsy is written in PHP, but there are already clients written in Python, Ruby, Java, etc. that are being contributed to the project. The key here is that collecting a new metric from our application code is one line of code.
We’re re-running all four of our talks at the Etsy office in Brooklyn, NY on Thursday, March 31 (sign up here). Come join us to hear all of the details that you can’t glean from just reading the slides.
If you can’t make it in person, we’ll be live streaming the event.