I'm pleased to finally release a side-project I've been whittling away at for a few months: Rack::Cache is a piece of Rack middleware that implements most of RFC 2616’s caching features with a basic set of storage options (disk, heap, and memcached) and a configuration system for tweaking cache policy. It should work equally well with any Rack-enabled Ruby web framework and is tested on Ruby 1.8.6 and 1.8.7.

high-level rack-cache diagram

Rack::Cache relies entirely on standard HTTP headers produced by your application. It has no application level caching API.

You need to understand HTTP caching.

The middleware piece sits near the front of each backend process and acts like a gateway proxy cache (e.g, Varnish, Squid) but without requiring the infrastructure investment of a separate daemon process. This approach is quite different from the integrated caching systems built into most Ruby web frameworks. There are pros and cons which I plan to explore with some depth in future posts here.

The basic goal is standards-based HTTP caching that scales down to the early stages of a project, development environments, light to medium trafficked sites, stuff like that. HTTP’s caching model is wildly under-appreciated in the Ruby web app community and my hope is that making its benefits more accessible will lead to wider understanding and acceptance.

Rack::Cache is still very much a work in progress but is far enough along to be useable (it’s serving this site, in fact). The following HTTP caching features are currently supported:

  • Expiration-based caching. Responses are served from cache while fresh without consulting the backend application. The Cache-Control: max-age=N and Expires response headers control a response’s freshness lifetime.

  • Validation. The cache stores responses even if no freshness information is present so long as there’s a cache validator (Last-Modified or ETag). Subsequent requests result in a conditional GET request to the application and if the stored response is unmodified, it’s served from cache. Your backend should never generate the same response twice.

  • Vary support.

There are a few notable things still missing.

  • Explicit purge / manual cache invalidation. There is currently no supported method for manually invalidating a cache entry.

  • Multi-thread. There’s nothing fundamentally hard about this, I just haven’t got around to testing in a threaded environment and working out the kinks.

  • Performance. The current focus is on nailing down a solid implementation of HTTP’s basic caching features and providing good documentation. I have not profiled the system or performed any benchmarks. Anecdotal evidence suggests that the performance situation is basically swell.

Finally, I'd like to note that Rack::Cache is based largely on the work of the internet standards community and that portions of its design were inspired by Django’s cache framework and Varnish’s configuration language (VCL). Huge thanks to everyone involved with any of this stuff.

This entry has been tagged web, ruby, http, coding, caching, rack-cache, rack — follow a tag for related essays, articles, and bookmarks.

Discuss

  1. Why not do cache invalidation using the HTTP DELETE method? Browsers can’t send it, but your backend can…

    — David Ulevitch on Saturday, October 25, 2008 at 11:30 AM #

  2. RFC2616 states w/r/t a DELETE request:

    “If the request passes through a cache and the Request-URI identifies one or more currently cached entities, those entries SHOULD be treated as stale. Responses to this method are not cacheable.”

    — David Ulevitch on Saturday, October 25, 2008 at 11:30 AM #

  3. David: it’s planned. Specifically, the PUT/POST/DELETE invalidation logic described in RFC 2616 Section 13.10.

    My long term goal is to conform to all MUSTs (and as many SHOULDs as possible) defined in Section 13 of RFC 2616.

    — Ryan Tomayko on Saturday, October 25, 2008 at 12:29 PM #

  4. Neat. Looks very promising. Interested in help?

    — Bob Aman on Saturday, October 25, 2008 at 07:45 PM #

  5. Bob: Absolutely. There’s a TODO list and I'm open to any and all features/bug-fixes that would bring the system further into line with RFC 2616. It’s very much still an experimental project so nothing’s totally out of the question. There should be enough code there to set the tone.

    I appreciate changes in the form of links to git repos with discrete commits and good commit messages but I'm also happy to accept patches via email. The easiest thing is probably just to fork me on GitHub.

    — Ryan Tomayko on Saturday, October 25, 2008 at 08:17 PM #

  6. Hey David, how did you create that neat diagram?

    — Manish on Sunday, October 26, 2008 at 12:56 AM #

  7. This is exciting! I've written a micro framework called Gossamer using Rack and memcached. I've been thinking lately of removing the explicit dependency on the cache, possibly by using a front-end cache like squid. But it seems that your Rack::Cache would get me most of the way there, and be more cleanly implemented in Rack — as well as being less of a deployment pain.

    — John Labovitz on Sunday, October 26, 2008 at 02:47 PM #

  8. Hey Ryan,

    It’s great to have more cache implementations; there are few around, mostly because it’s actually a fairly complex thing to implement well.

    However, I was somewhat surprised by your statement “…without the overhead of a separate daemon process.” To me, this is the fundamental limitation of this kind of of approach (whether it be in Python with WSGI or Django, or here in Ruby); concurrency.

    Any modern Web proxy/cache implementation is built to be massively concurrent, based upon an event loop. The limited functions of a proxy/cache are actually a huge win in this respect; because they only ever need block on disk access and network access — both things that can be factored out on modern OSs — it’s possible to make assumptions in your architecture and get massive scalability and performance wins.

    In contrast, getting good concurrency in a dynamic language takes a tremendous amount of programmer discipline, and even then it’s very difficult to get performance and scalability close to an order of magnitude less* than the approach above. And, of course, if you’re running on Apache — as most people will be — you pretty much throw any serious amount of concurrency out of the window.

    So, the “the overhead of a separate daemon process” is IMO a mistaken characterisation. In my day job we run a lot of servers this way; dynamic code on the back end, fronted by a Squid process to speed things up. It works — and scales — remarkably well.

    By no means do I want to discourage you from taking this route — as I said, it’s great to have more implementations — but realistically, it will never perform as well as a separate Squid, Varnish, etc. OTOH, it will be easier to integrate new features into your cache; we've been doing that progressively in Squid for a while now, because it didn’t meet a lot of our needs, but of course it’s a slow process.

    Cheers,

    * good concurrency != threads

    — Mark Nottingham on Sunday, October 26, 2008 at 07:02 PM #

  9. Hi Mark. I was hoping you would stop by.

    Let me clarify: when I said “overhead”, I was speaking specifically about the time and research involved in getting a separate caching daemon up and running and then managing it and not about the machine resource overhead involved. I should have been more clear on this.

    As you mentioned, a separate, high performance gateway cache has massive performance and resource use advantages compared to something like Rack::Cache. Unfortunately, I'm afraid the complications involved in scaling these systems down is hurting adoption of more advanced HTTP caching techniques because the benefits are unattainable without the investment (headache) of managing another large piece of infrastructure. So caching becomes an afterthought – something you learn about only after you've made the investment in a high performance gateway cache. Rack::Cache aims to make the benefits of HTTP caching available very early and broadly, even if in a diminished capacity. I suppose its something of a worse is better approach.

    I do think the performance improvements created by something like Rack::Cache can be extremely compelling for the types of web applications I've been dealing with for the past few years. A typical Ruby web app performs lots of database reads (often with multiple round trips) and heavy string manipulation. The sophisticated web and ORM frameworks bring large productivity gains and ease maintenance but there’s a lot of extra code to move through and the layers of abstraction often sacrifice performance for simplicity. On top of all that, we’re running on one of the slowest interpreters in existence.

    What all that adds up to is significant resource (especially CPU and memory) use in generating a full response. I'm finding that, in a great many cases, I can generate cache validators in 1/10th – 1/50th of the time required to generate a full response due to less DB interaction, string operations, and just spending less time in the framework. With something as simple as Rack::Cache, it’s possible to build my app such that my backend never has to generate the same response twice (within resource limits, of course).

    None of these arguments support Rack::Cache over a high performance daemon or CDN. But Rack::Cache makes the benefits available in some form for much less cost. And the beautiful thing is that your app has been built with standards from the beginning so moving up to a high performance system should not involve major application-wide changes to your backend codebase. The systems are complimentary.

    Also, systems like Rack::Cache might not make sense in other environments. The value of such a system is directly proportional to the average amount of resource use required to generate a full response compared to that required to generate cache validators. A PHP app that performs direct, hand-crafted SQL queries and doesn’t use a template engine might not see the same increases because the difference between generating full responses and validators is negligible. But a Ruby app that uses an ORM and makes tens of thousands of method calls per request is just ripe for massive gains because generating cache validators is significantly less work than generating a full response.

    — Ryan Tomayko on Sunday, October 26, 2008 at 09:31 PM #

  10. Ryan,

    I totally agree that for most devs getting a cache like Squid up and running is a difficult, unappealing task, for the reasons you state (also, Squid for one isn’t too friendly to typical Web devs, as its core audience is sysadmins and network folk). While we can work on these platforms to make them easier to understand and work with (I've been dipping my toes in here), it’s still a big leap for most.

    One of the problems that I've had in getting caching adopted is that by the time people think of it, it’s too late; they've already designed and coded their resources. The fact that it’s hard to get them a cache to work with from the get go only complicates this.

    So, making caching easier to integrate into apps from the beginning — by building HTTP caching in to the frameworks — is a fantastic, very smart strategy to make sure that when people need that high-end scalability, they can transition easily, by interposing an intermediary (still one of the great strengths of HTTP IMO).

    Thanks for the clarification (I only saw technical overhead, not social/management overhead when I read that), and kudos!

    I'm also heartened by the IMS/INM validation support that’s starting to show up; earlier frameworks tended to do the simple thing and just compare a hash of the content after generating it, thereby incurring all of the cost still. Newer ones (including this, AFAICT) are being much smarter and allowing validation to “see” inside of application semantics to shortcut that.

    Cheers,

    — Mark Nottingham on Monday, October 27, 2008 at 01:30 AM #

  11. P.S. Want to work on generic HTTP caching test suites?

    — Mark Nottingham on Monday, October 27, 2008 at 01:31 AM #

  12. Yes, I'd very much like to work on a generic HTTP caching test suite. Rack::Cache has a decent set of unit tests but I really wish there was something that was external and operated at the wire protocol level.

    Also, I've edited the post to be more clear on the trade-off between Rack::Cache and a high-performance intermediary (s/overhead/infrastructure investment/).

    — Ryan Tomayko on Monday, October 27, 2008 at 10:28 AM #

  13. +1 to the generic HTTP caching test suite.

    — Bob Aman on Wednesday, October 29, 2008 at 07:46 PM #

Leave a comment





(syntax: markdown)