Ryan Tomayko

Show How, Don't Tell What - A Management Style

2012-04-02T00:00:00-07:00

One of the things I'm most excited about working at GitHub is the opportunity to take our time and think through organization and process problems from first principle instead of blindly copying other companies or adopting status quo approaches developed in the last century. We're beholden to no one except the good people that pay us for our products and that gives us the freedom to build a company optimized for delivering the best experience - whatever it takes.

Last year, as GitHub began to grow rapidly, I was promoted to Director of Engineering. That makes me a manager of sorts. Gross, right? Actually, it's turned out not to be very horrible at all. Like most things at GitHub, I was given complete control and encouraged and expected to define the role in whatever way made most sense to me. I want to share some of what I've come up with.

Don't tell people what to do

I've never actually told anyone what to do here. In fact, I vehemently refuse to tell people what to do. Here are just a couple reasons why:

I don't scale. If I tell someone what to do and they do it, then what? Do I have to tell them another thing to do? What happens when I have to decide what to do for 20 people?
Telling people what to do is lazy. Instead, try to convince them with argument. This is how humans interact when there's no artificial authority structure and it works great. If you can't convince people through argument then maybe you shouldn't be doing it.

So if I don't tell people what to do, then what purpose do I serve and what exactly do I do all day?

I show people how to plan, build, and ship product together.

Essentially, I try to create little mini-managers, each responsible for managing a single person: their self. At first this is mainly about teaching people how to figure out what to work on right now (prioritization). Then it's more a matter of building their confidence both in themselves and their team to a point where they just do things they know is right without even talking to me (autonomy).

Lead by example as loud as possible

I actually don't show people how to make decisions and ship product in any real direct way. There's no How To Ship Product training class or anything like that. Instead, I just do work.

I write down ideas and then market them internally. I ask designers about their comps and concept work. I write code with kick ass docs and tests, sometimes while building out the backend for a feature and sometimes just to clean shit up because it needs it. I deploy responsibly because site stability is job number zero. I soft ship new features and try to get other employees to use them. I write and review blog posts and ship features. I fix bugs. I work with support.

That's just how you ship software product in 2012.

The only thing that may be unique and interesting about the way I do it is that I insist on being extremely visible throughout the entire process. That means removing sidebar conversations, private meetings, face time, IM, private pairing, and anything else that limits the visibility of work and process.

Just forcing people to be open and electronic where I'm concerned goes a long way in keeping things managed because work at every stage of the product cycle is in everyones face all the time. For one, people tend to self manage when everyone else can see what they're doing. Also, other people jump in when they notice something amiss. Finally, everyone learns when anyone makes a mistake or does something brilliant.

If a "management style" is something I have, this openness doctrine would be the cornerstone.

Get people contributing

When someone is new to the process at GitHub, I make a concious effort to @mention them on issues, comments, and commits to bring them into a bunch of different projects and discussions.

When they first have an idea or feedback on something, I turn it around on them and use the oldest trick from the open source playbook:

"Where's your patch?"

In other words, go build the thing in your head, show me how you got there, and then sell it to me and all your peers.

I review their pull requests and try to imagine how they must be perceiving the process, our product, our customers. When I think their perception is off, I say things to correct it. When I think they're really getting it, I say things to let them know they're on the right track.

Make everyone a manager

It's often cited that GitHub doesn't have managers. In my opinion, a better way to describe the phenomenon would be to say that everyone at GitHub is a manager. Instead of assigning 100% management duties to individuals, the basic role of management is spread between 1.) every single employee, and 2.) a set of custom in-house tools that serve to keep everyone in the know with regards to other projects.

This approach to management has real tangible benefits. Imagine 100% of your "workforce" being able to operate at maximum productivity / efficiency without oversight. The only real requirement for work to be done is for people to be sober (something we're working on).

Imagine 100% of your "workforce" understanding every part of the product delivery cycle and being able to make critical path decisions on the spot.

Imagine what would happen if 100% of your "workforce" were "decision makers".

These are my goals right now.

Take a vacation

Today, I start back at work after 10 days on family vacation. This is the first substantial bit of time I've had away since I started with GitHub almost three years ago.

Guess how many meetings I scheduled to make sure projects stayed on track while I was away? Zero.

I sent a blanket email the day before I left and asked for last minute concerns. I received a single reply:

Subject: Re: Hawaii
From: Chris Wanstrath <chris@github.com>
To: Ryan Tomayko <ryan@github.com>
Date: Thu, 22 Mar 2012 17:22:20 -0700

On Thu, Mar 22, 2012 at 2:37 PM, Ryan Tomayko <ryan@github.com> wrote:
> Guys, I'm heading to Hawaii tomorrow. I'll be there for a week and
> plan to be as AFK as possible. If there's anything pressing I owe you
> right now or stuff that'd help in my absence please let me know ASAP
> so I can move it up my list of shit to get done before I peace out.

Have fun.

GitHub had one of its most productive weeks in recent memory. I relaxed and had fun on the beach.

The one downside I've found in all this is that I'm apparently extremely dispensable at this point. Shrug. Maybe I should take more vacations.

(comment)

AWK-ward Ruby

2011-04-26T00:00:00-07:00

This essay was to be published as a companion piece to The Shell Hater's Handbook, an introductory talk on UNIX shell programming for Ruby hackers given at GoGaRuCo 2010. Alas, the post-conference wrap up magazine will not be published this year and so I'm making the essay available here instead.

Ruby, like most successful languages, was assembled from pieces of things that came before it: Smalltalk's consistent object system, Perl's practical syntax, UNIX's sensibilities. Not that it didn't bring entirely new innovations (~~block syntax!~~ Smalltalk had block syntax first!) of its own, but it's amazing to consider how much of Ruby's design rests on the elegant packaging of old concepts into a new coherent whole.

There's something less obvious but perhaps more essential that Ruby borrowed: the very concept of blatant, unashamed borrowing. In his 1999 talk, Perl, the first postmodern computer language, Larry Wall states plainly that Perl was built mostly from things that "didn't suck" in the languages that preceded it:

When I started designing Perl, I explicitly set out to deconstruct all the computer languages I knew and recombine or reconstruct them in a different way, because there were many things I liked about other languages, and many things I disliked. I lovingly reused features from many languages. (I suppose a Modernist would say I stole the features, since Modernists are hung up about originality.) Whatever the verb you choose, I've done it over the course of the years from C, sh, csh, grep, sed, awk, Fortran, COBOL, PL/I, BASIC-PLUS, SNOBOL, Lisp, Ada, C++, and Python. To name a few. To the extent that Perl rules rather than sucks, it's because the various features of these languages ruled rather than sucked.

Ruby, the story goes, borrowed much from Perl: integral regular expressions, statement modifiers (do_this if that), array/hash literals, funny global variable names, and of course the philosophy of having more than one way to do the same thing (TMTOWTDI).

Or did it?

If these features didn't originate with Perl, as Wall seems to imply, then where did they come from?

One of the most important influences on Perl's design was AWK. So much so that Perl was sometimes described as a semantic superset of AWK. Are the relics of AWK still present in Ruby? Let's see.

Today, AWK is probably best known as a versatile tool for extracting fields from delimited flat files in a shell pipeline:

cat /etc/passwd | awk -F: '{ print $1 }'

It's rare to see AWK used for more complex problems in modern systems, but there's actually a full blown programming language lurking beneath the surface. It was at one time used to solve a lot of the same problems people commonly use Ruby, Perl, or Python to solve today.

You might find some of AWK's language features familiar:

Associative array type.
Automatic string, integer, and floating point value types.
C-style if, while, and do constructs.
For-each style for construct for iterating over associative arrays.
Arithmetic (+, -, *, /), modulu-division (%), exponentiation (^), increment/decrement (++, --), and assignment shorthand (+=, -=, *=, ...) operators.
Array membership operator (expr in array).
Integral regular expression type and matching operators (str ~ /pattern/).
Comprehensive builtin function library (a small sample: printf, gsub, split, substr, cos, sin, log, sqrt).
User defined functions.

Not bad for 1977.

It would seem that a large portion of Ruby's basic syntax and semantics were present in AWK. So how did Perl come to dominate the problem space? There must be something very different about AWK.

While AWK had much of the primitive syntax right, it also overcompensated for a specific case: processing streams of delimited text. The top-level context is used exclusively for declaring one or more matching statements:

pattern { action }
...

Here, pattern is a full blown expression and action is a block of code executed when pattern evaluates truthfully. The pattern is tested for each line (or record) of input and action is executed when pattern returns truthfully. Omitting the pattern causes the action to be executed for every line.

There's special patterns for setting actions up to run before the first line of input is read and after all lines have been processed. Here's an example that uses the special BEGIN pattern along with a regular expression match. It prints all the usernames from /etc/passwd while avoiding comment lines:

cat /etc/passwd |
awk '
    BEGIN     { FS = ":" }
    /^[a-z_]/ { print $1 }
'

(NOTE: You can paste bomb that into your shell on just about any UNIX system.)

Here's a more complex example that shows off some of AWK's advanced features, like associative arrays and for-in syntax. It calculates word frequencies from the text of Jonathan Swift's, A Modest Proposal:

curl -s http://www.gutenberg.org/files/1080/1080.txt |
awk '
    BEGIN { FS="[^a-zA-Z]+" }

    {
        for (i=1; i<=NF; i++) {
            word = tolower($i)
            words[word]++
        }
    }

    END {
        for (w in words)
             printf("%3d %s\n", words[w], w)
    }
' |
sort -rn

It may seem strange, but this style of programming was very common in UNIX's hayday. Instead of programs being dominated by a single language like Perl or Ruby, you'd build pipelines that combined standard utilities (like sort shown above), sprinkle in bits and pieces of AWK as needed, and drop down to C when performance was critical.

Perl took the guts of AWK and left behind the mandatory pattern matching at the top-level. That simple design change turned what was a special purpose language for processing delimited text streams into what we know today as a general purpose scripting language.

But that's not the end of the story.

It was important that Perl be able to act as a replacement for AWK in all its capacities, including within shell pipelines. This meant having the ability to run perl in a kind of top-level AWK mode. Ruby borrowed this capability from Perl, making it possible to use Ruby for the same style of programming facilitated by AWK, complete with BEGIN and END blocks!

Here's the word frequency script in AWK-ish Ruby:

curl -s http://www.gutenberg.org/files/1080/1080.txt |
ruby -ne '
  BEGIN { $words = Hash.new(0) }

  $_.split(/[^a-zA-Z]+/).each { |word| $words[word.downcase] += 1 }

  END {
    $words.each { |word, i| printf "%3d %s\n", i, word }
  }
' |
sort -rn

The -n argument causes Ruby to assume a while gets(); ... end loop around the provided script. $_ is set to the last line read and the BEGIN and END blocks function exactly as they did in AWK.

I like Unicorn because it's Unix

2009-10-06T00:00:00-07:00

Eric Wong's mostly pure-Ruby HTTP backend, Unicorn, is an inspiration. I've studied this file for a couple of days now and it's undoubtedly one of the best, most densely packed examples of Unix programming in Ruby I've come across.

Unicorn is basically Mongrel (including the fast Ragel/C HTTP parser), minus the threads, and with teh Unix turned up to 11. That means processes. And all the tricks and idioms required to use them reliably.

We're going to get into how Unicorn uses the OS kernel to balance connections between backend processes using a shared socket, fork(2), and accept(2) -- the basic Unix prefork model in 100% pure Ruby.

But first ...

A note about Unix programming in Ruby in general

We should be doing more of this. A lot more of this. I'm talking about fork(2), execve(2), pipe(2), socketpair(2), select(2), kill(2), sigaction(2), and so on and so forth. These are our friends. They want so badly just to help us.

NOTE The links above are to Linux section 2 manpages; in many cases, BSD manpages are vastly superior.

Ruby, Python, and Perl all have fairly complete interfaces to common Unix system calls as part of their standard libraries. In most cases, the method names and signatures match the POSIX definitions exactly. Yet, of the groups, only the Perl people seem to regularly (and happily) apply common Unix idioms to a wide range of problem areas.

Unix is not one of the "perlisms" Ruby should be trying to distance itself from. Perl got that part right. And with immediately recognizable Unix system calls spewed all over the core library, Ruby feels like it was built for Unix hacking. It's surprising to see how infrequently this stuff is used as intended or even talked about.

man 2 intro

Documentation is likely part of the problem. Here's a small sample of Ruby core docs on an assortment of Unix system calls -- the kind we don't use enough:

Process::fork - oh, sorry, see Kernel::fork ... pitiful.
Kernel::exec - really?
Process::kill - thanks!
IO::pipe - not bad.
Socket::socketpair - entirely undocumented, sorry.

That Ruby provides no useful documentation isn't actually a problem if you happen to have experience programming Unix. Then you would of course just happen to know that there's a secret manual section with extensive reference information on each of these commands.

Here's a snippet of the (BSD) manpage for pipe(2) as shipped with MacOS X (also: Linux version):

$ man 2 pipe
PIPE(2)                     BSD System Calls Manual                    PIPE(2)

NAME
     pipe -- create descriptor pair for interprocess communication

SYNOPSIS
     #include <unistd.h>

     int
     pipe(int fildes[2]);

DESCRIPTION
     The pipe() function creates a pipe (an object that allows unidirectional
     data flow) and allocates a pair of file descriptors.  The first descriptor
     connects to the read end of the pipe; the second connects to the write end.

     Data written to fildes[1] appears on (i.e., can be read from) fildes[0].
     This allows the output of one program to be sent to another program: the
     source's standard output is set up to be the write end of the pipe; the
     sink's standard input is set up to be the read end of the pipe.  The pipe
<snip>

Most Ruby developers probably don't have a background in Unix C programming. How are they supposed to know that all those undocumented parts of the standard library are undocumented because -- DUH! -- you just have to go enter man 2 THING into the console. Obviously.

Threads are out

There's another problem with Unix programming in Ruby that I'll just touch on briefly: Java people and Windows people. They're going to tell you that fork(2) is bad because they don't have it on their platform, or it sucks on their platform, or whatever, but it's cool, you know, because they have native threads, and threads are like, way better anyways.

Fuck that.

Don't ever let anyone tell you that fork(2) is bad. Thirty years from now, there will still be a fork(2) and a pipe(2) and a exec(2) and smart people will still be using them to solve hard problems reliably and predictably, just like they were thirty years ago.

MRI Ruby people need to accept, like Python (you have seen multiprocessing, yes?), that Unix processes are one of two techniques for achieving reliable concurrency and parallelism in server applications. Threads are out. You can use processes, or async/events, or both processes and async/events, but definitely not threads. Threads are out.

Anyway, Unicorn.

Unicorn, and preforking servers in general, create a listening socket in a parent process and then fork off one or more child processes, each of which calls accept(2) on the same shared listening socket. The kernel manages the task of distributing connections between accepting processes.

Let's start with a simplified example. A simple echo server that balances connections between three child processes:

# simple preforking echo server in Ruby
require 'socket'

# Create a socket, bind it to localhost:4242, and start listening.
# Runs once in the parent; all forked children inherit the socket's
# file descriptor.
acceptor = Socket.new(Socket::AF_INET, Socket::SOCK_STREAM, 0)
address = Socket.pack_sockaddr_in(4242, 'localhost')
acceptor.bind(address)
acceptor.listen(10)

# Close the socket when we exit the parent or any child process. This
# only closes the file descriptor in the calling process, it does not
# take the socket out of the listening state (until the last fd is
# closed).
#
# The trap is guaranteed to happen, and guaranteed to happen only
# once, right before the process exits for any reason (unless
# it's terminated with a SIGKILL).
trap('EXIT') { acceptor.close }

# Fork you some child processes. In the parent, the call to fork
# returns immediately with the pid of the child process; fork never
# returns in the child because we exit at the end of the block.
3.times do
  fork do
    # now we're in the child process; trap (Ctrl-C) interrupts and
    # exit immediately instead of dumping stack to stderr.
    trap('INT') { exit }

    puts "child #$$ accepting on shared socket (localhost:4242)"
    loop {
      # This is where the magic happens. accept(2) blocks until a
      # new connection is ready to be dequeued.
      socket, addr = acceptor.accept
      socket.write "child #$$ echo> "
      socket.flush
      message = socket.gets
      socket.write message
      socket.close
      puts "child #$$ echo'd: '#{message.strip}'"
    }
    exit
  end
end

# Trap (Ctrl-C) interrupts, write a note, and exit immediately
# in parent. This trap is not inherited by the forks because it
# runs after forking has commenced.
trap('INT') { puts "\nbailing" ; exit }

# Sit back and wait for all child processes to exit.
Process.waitall

Run that with ruby echo.rb and then connect a few times with netcat:

$ echo "hello world" | nc localhost 4242
child 86900 echo> hello world

$ echo "hello world" | nc localhost 4242
child 86901 echo> hello world

$ echo "hello world" | nc localhost 4242
child 86902 echo> hello world

...

Use telnet if you don't have netcat:

$ telnet localhost 4242
telnet localhost 4242
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
child 86902 echo> hello world
hello world

$ telnet localhost 4242
...

This isn't exactly what Unicorn does but it nicely illustrates one of the basic concepts underlying the design. This technique can be used for simple network servers as above, or it could be used for general IPC-based balanced multiprocessing. For example, you could connect directly to the shared socket in the parent process to distribute work between a set of child processes.

Unicorns do it with `select(2)`

Instead of a blocking accept(2), Unicorn uses a blocking select(2) with an error pipe and a timeout so that it can bust out and do some other basic housekeeping, like reopening logs, processing signals, and maintaining a heartbeat with the master process.

You can watch this play out in Unicorn::HttpServer#worker_loop, the heart of each Unicorn child/worker process. Most notable is the following call to select(2):

ret = IO.select(LISTENERS, nil, SELF_PIPE, timeout) or redo

This blocks until one of three things happen:

A connection is pending on the shared listening socket (passed in the LISTENERS array) and is ready to be dequeued by accept(2), in which case the ready socket is returned.
Some notable error state occurs on the file descriptor in SELF_PIPE (like when it's closed), in which case the child's side of the pipe is returned as an IO object. This really deserves its own essay, but I'll take a quick shot: the IO object in SELF_PIPE is created in the parent process with pipe(2) (IO.pipe) before the children are forked off. The children then write on the pipe to achieve basic one-way IPC between child and master. It's used here in the call to select(2) to detect the master going down unexpectedly - parent death causes the pipe to close. Unicorn children go down fast when their master dies.
The timeout elapses.

If select(2) returns due the first condition, the child process calls Socket#accept_nonblock on the shared listening socket, which either returns a newly established connection or fails with Errno::EAGAIN, signaling that some other child process beat us to the accept(2). In either case, accept returns immediately and does not block.

This is just one of many beautiful Unix idioms you'll find in Unicorn. The signal handling, hot process replacement/reloading with rollback, the SELF_PIPE IPC technique, and the fchmod(2) based heartbeat implementation are at least as interesting. Check it out.

Comments will be broken here for a while. There's a discussion brewing on Hacker News. Also, if you have examples of the prefork echo server in other languages, like Jacob's Python implementation, shoot me a link via mail or twitter and I'll link to it here, from my linkings, and on Twitter.

HTTP Caching Talk at RailsConf '09

2009-04-27T00:00:00-07:00

I'm doing a talk on HTTP caching at this year's RailsConf in Vegas, bright and early (9:45 AM), on the last day of the conference. I'm assuming you'll be tired after a week's worth of hard nights and depressed having lost all your money, but come by anyway.

I've had ideas for a killer general introduction to modern HTTP caching for years but was hesitant to give it since the set of available gateway caches (Squid, basically) were not what I'd consider practical for the audience I wanted to target (all ruby web developers). A lot has changed within the past year: Rack::Cache was released, Heroku put a high performance gateway cache / accelerator in front of every app, most web frameworks provide utilities for basic HTTP caching stuff, Apache's mod_cache is much improved, Varnish is seeing a lot of adoption -- it's time to start talking about this stuff.

And even more has happened in the months since I submitted my original proposal. I caught a recording of the RailsEnvy guys's acts_as_conference 09 presentation, wherein Gregg Pollack utterly nails a huge portion of what I'd hoped to speak about. He followed that up with a highly polished screencast (if you can call it a "screencast" -- he uses a goddam green screen) on Advanced HTTP Caching, which is more or less the talk I planned to give at RailsConf.

So I've decided to take all of this as an opportunity and am shifting my talk around to explore some of the more advanced and experimental stuff people are doing with HTTP caching. Not everything is worked out yet but I'm happy with the direction and excited at the prospect of having an audience for some of these ideas sooner than later. I still plan to run through the basics but quickly, and only enough to set up the latter part of the presentation. I strongly recommended watching Gregg's piece if you plan on attending.

Hope to see you there.

UPDATE: I wasn't able to deliver on the advanced stuff due to time constraints but I think the talk went okay. Huge thanks to everyone that attended and here's the slides for those that weren't able to make it.

Why "require 'rubygems'" Is Wrong

2009-01-27T00:00:00-08:00

The following was originally published at gist.github.com/54177 but has been copied here for posterity and because people keep asking me on IRC for the original link; it must be hard to find on gist for some reason.

In response to all the "huh?" responses to this tweet: You should never do this in a source file included with your library, app, or tests:

require 'rubygems'

The system I use to manage my $LOAD_PATH is not your library / app / tests concern. Whether rubygems is used or not is an environment issue. Your library or app should have no say in the matter. Explicitly requiring rubygems is either not necessary or misguided.

When you feel the urge to "require 'rubygems'" because some shit isn't working, stop and consider whether one of these solutions is more appropriate:

1. RubyGems installs special wrapper versions of executables included with gems. Here's /opt/local/bin/rake on my system as an example:

$ cat /opt/local/bin/rake
#!/opt/local/bin/ruby
#
# This file was generated by RubyGems.
#
# The application 'rake' is installed as part of a gem, and
# this file is here to facilitate running it.
#
require 'rubygems'
version = ">= 0"
if ARGV.first =~ /^_(.*)_$/ and Gem::Version.correct? $1 then
  version = $1
  ARGV.shift
end
gem 'rake', version
load 'rake'

In this case, rubygems is required when the executable is run. Explicitly requiring rubygems in code loaded by these files doesn't make sense. The whole point of these wrapper scripts is to push the rubygems loading machinery into the environment and out of your app / library / tests.

2. When running ruby FOO.rb and a LoadError occurs because rubygems is not available, DO NOT add require 'rubygems' to FOO.rb. Instead, run the command as ruby -rubygems FOO.rb. This will require rubygems before evaluating FOO.rb.

3. When running ruby FOO.rb and a LoadError occurs because rubygems is not available, DO NOT add require 'rubygems' to FOO.rb. Instead, set the RUBYOPT environment variable:

$ RUBYOPT="rubygems"
$ export RUBYOPT
$ ruby FOO.rb

You can even put that in your ~/.{bash,zsh,sh}rc if you prefer to always have rubygems loaded and available.

Why You Shouldn't Force Rubygems On People

When I use your library, deploy your app, or run your tests I may not want to use rubygems. When you require 'rubygems' in your code, you remove my ability to make that decision. I cannot unrequire rubygems, but you can not require it in the first place.

Things Caches Do

2008-11-16T00:00:00-08:00

There are different kinds of HTTP caches that are useful for different kinds of things. I want to talk about gateway caches -- or, "reverse proxy caches" -- and consider their effects on modern, dynamic web application design.

Draw an imaginary vertical line, situated between Alice and Cache, from the very top of the diagram to the very bottom. That line is your public, internet facing interface. In other words, everything from Cache back is "your site" as far as Alice is concerned.

Alice is actually Alice's web browser, or perhaps some other kind of HTTP user-agent. There's also Bob and Carol. Gateway caches are primarily interesting when you consider their effects across multiple clients.

Cache is an HTTP gateway cache, like Varnish, Squid in reverse proxy mode, Django's cache framework, or my personal favorite: rack-cache. In theory, this could also be a CDN, like Akamai.

And that brings us to Backend, a dynamic web application built with only the most modern and sophisticated web framework. Interpreted language, convenient routing, an ORM, slick template language, and various other crap -- all adding up to amazing developer productivity. In other words, it's horribly slow and bloated... and awesome! There's probably many of these processes, possibly running on multiple machines.

(One would typically have a separate web server -- like Nginx, Apache or lighttpd -- and maybe a load balancer sitting in here as well but that's largely irrelevant to this discussion and has been omitted from the diagrams.)

Expiration

Most people understand the expiration model well enough. You specify how long a response should be considered "fresh" by including either or both of the Cache-Control: max-age=N or Expires headers. Caches that understand expiration will not make the same request until the cached version reaches its expiration time and becomes "stale".

A gateway cache dramatically increases the benefits of providing expiration information in dynamically generated responses. To illustrate, let's suppose Alice requests a welcome page:

Since the cache has no previous knowledge of the welcome page, it forwards the request to the backend. The backend generates the response, including a Cache-Control header that indicates the response should be considered fresh for ten minutes. The cache then shoots the response back to Alice while storing a copy for itself.

Thirty seconds later, Bob comes along and requests the same welcome page:

The cache recognizes the request, pulls up the stored response, sees that it's still fresh, and sends the cached response back to Bob, ignoring the backend entirely.

Note that we've experienced no significant bandwidth savings here -- the entire response was delivered to both Alice and Bob. We see savings in CPU usage, database round trips, and the various other resources required to generate the response at the backend.

Validation

Expiration is ideal when you can get away with it. Unfortunately, there are many situations where it doesn't make sense, and this is especially true for heavily dynamic web apps where changes in resource state can occur frequently and unpredictably. The validation model is designed to support these cases.

Again, we'll suppose Alice makes the initial request for the welcome page:

The Last-Modified and ETag header values are called "cache validators" because they can be used by the cache on subsequent requests to validate the freshness of the stored response without requiring the backend to generate or transmit the response body. You don't need both validators -- either one will do, though both have pros and cons, the details of which are outside the scope of this document.

So Bob comes along at some point after Alice and requests the welcome page:

The cache sees that it has a copy of the welcome page but can't be sure of its freshness so it needs to pass the request to the backend. But, before doing so, the cache adds the If-Modified-Since and If-None-Match headers to the request, setting them to the original response's Last-Modified and ETag values, respectively. These headers make the request conditional. Once the backend receives the request, it generates the current cache validators, checks them against the values provided in the request, and immediately shoots back a 304 Not Modified response without generating the response body. The cache, having validated the freshness of its copy, is now free to respond to Bob.

This requires a round-trip with the backend, but if the backend generates cache validators up front and in an efficient manner, it can avoid generating the response body. This can be extremely significant. A backend that takes advantage of validation need not generate the same response twice.

Combining Expiration and Validation

The expiration and validation models form the basic foundation of HTTP caching. A response may include expiration information, validation information, both, or neither. So far we've seen what each looks like independently. It's also worth looking at how things work when they're combined.

Suppose, again, that Alice makes the initial request:

The backend specifies that the response should be considered fresh for sixty seconds and also includes the Last-Modified cache validator.

Bob comes along thirty seconds later. Since the response is still fresh, validation is not required; he's served directly from cache:

But then Carol makes the same request, thirty seconds after Bob:

The cache relies on expiration if at all possible before falling back on validation. Note also that the 304 Not Modified response includes updated expiration information, so the cache knows that it has another sixty seconds before it needs to perform another validation request.

The basic mechanisms shown here form the conceptual foundation of caching in HTTP -- not to mention the Cache architectural constraint as defined by REST. There's more to it, of course: a cache's behavior can be further constrained with additional Cache-Control directives, and the Vary header narrows a response's cache suitability based on headers of subsequent requests. For a more thorough look at HTTP caching, I suggest Mark Nottingham's excellent Caching Tutorial for Web Authors and Webmasters. Paul James's HTTP Caching is also quite good and bit shorter. And, of course, the relevant sections of RFC 2616 are highly recommended.

(Oh, and the diagrams were made using websequencediagrams.com, a very simple, text-based sequence diagram generating web service thingy.)

Introducing Rack::Cache

2008-10-24T00:00:00-07:00

I'm pleased to finally release a side-project I've been whittling away at for a few months: Rack::Cache is a piece of Rack middleware that implements most of RFC 2616's caching features with a basic set of storage options (disk, heap, and memcached) and a configuration system for tweaking cache policy. It should work equally well with any Rack-enabled Ruby web framework and is tested on Ruby 1.8.6 and 1.8.7.

Rack::Cache relies entirely on standard HTTP headers produced by your application. It has no application level caching API.

You need to understand HTTP caching.

The middleware piece sits near the front of each backend process and acts like a gateway proxy cache (e.g, Varnish, Squid) but without requiring the infrastructure investment of a separate daemon process. This approach is quite different from the integrated caching systems built into most Ruby web frameworks. There are pros and cons which I plan to explore with some depth in future posts here.

The basic goal is standards-based HTTP caching that scales down to the early stages of a project, development environments, light to medium trafficked sites, stuff like that. HTTP's caching model is wildly under-appreciated in the Ruby web app community and my hope is that making its benefits more accessible will lead to wider understanding and acceptance.

Rack::Cache is still very much a work in progress but is far enough along to be useable (it's serving this site, in fact). The following HTTP caching features are currently supported:

Expiration-based caching. Responses are served from cache while fresh without consulting the backend application. The Cache-Control: max-age=N and Expires response headers control a response's freshness lifetime.
Validation. The cache stores responses even if no freshness information is present so long as there's a cache validator (Last-Modified or ETag). Subsequent requests result in a conditional GET request to the application and if the stored response is unmodified, it's served from cache. Your backend should never generate the same response twice.
Vary support.

There are a few notable things still missing.

Explicit purge / manual cache invalidation. There is currently no supported method for manually invalidating a cache entry.
Multi-thread. There's nothing fundamentally hard about this, I just haven't got around to testing in a threaded environment and working out the kinks.
Performance. The current focus is on nailing down a solid implementation of HTTP's basic caching features and providing good documentation. I have not profiled the system or performed any benchmarks. Anecdotal evidence suggests that the performance situation is basically swell.

Finally, I'd like to note that Rack::Cache is based largely on the work of the internet standards community and that portions of its design were inspired by Django's cache framework and Varnish's configuration language (VCL). Huge thanks to everyone involved with any of this stuff.