Interesting looking HTTP client library for Ruby with support for HTTP caching (with pluggable backends), basic and digest auth, intelligent redirect handling. It’s been around for a while and looks like it could eventually become similar in feature set to Python’s httplib2.
Finally, a sane looking sanitization lib that doesn’t try to do too much.
Christoffer Sawicki has started in on a partial implementation of ESI (Edge Side Includes) as a Rack middleware component. Put something like this in front of Rack::Cache and things get real interesting real fast :)
Markus Prinz with a nice review of important Ruby 1.9 changes.
“While I do consider the adjective ‘baroque’ to be a compliment, I must point out that Perl is actually more of a romantic piece, with allusions to various classical motifs. My favorite composer is Mahler, which should surprise no one.” — Larry Wall
Ian McKeller shows how easy it is to find web API “secret keys” when the user has access to the (network) client code. It’s actually a nice little crash coarse in how to write cracking software (here “crack” means warez scene type “crack”). That crazy shit like this is possible is why I got into software in the first place. Completely
Funniest thing I’ve seen on Joel on Software in quite some time.
I’ve annotated RFC 2616 Section 13 with details on where Rack::Cache is and isn’t compliant. Anything not highlighted should work as described in the RFC. I think I’ll be using SharedCopy more in the future.
Adam Wiggins: “gemweight.rb is a script to calculate the memory use and load time of a given gem.”
I’ve run numbers (very adhoc) on various gems before and I’m always surprised at the results. Libraries you’d think would be small are sometimes big and libraries you’d think would be big are almost always big :)
Via Ted Han (@knowtheory): “Hypertext allows information to be organized and connected in a variety of ways that provide the user with a flexible working environment. The following advantages highlight the benefits of working and learning in a hypertext environment…”
Interesting looking prefpane for MacOS X that tweaks some sleep settings:
“Just ‘sleep’ means that the notebook will go to sleep fast, but you loose the ability to change the battery. Just ‘sleep and hibernate’ will wake the computer fast, but sleeping will take ages as the contents of the memory are saved to disk before entering the sleep. MacOS uses ‘sleep and hibernate’ all the time by default. SmartSleep lets your notebook just ‘sleep’ while the battery has a high level. If the battery level drops below a certain point ( default is less then 20% or 20 minutes ) it will switch to ‘sleep and hibernate’. So you have the best of both worlds.”
“If you’re still terrified by the idea of talking to strangers, challenge yourself to talk to one stranger a day, every day, for 30 days. If you’re walking past someone on the sidewalk, say ‘Hi’, and the person looks at you and keeps walking (done that many times), your job is done for the day. If you walk up to a girl in a club and say ‘Hey!’, and she responds, with a slightly grossed out look ‘I have a boyfriend.’, congratulations, you’re one step closer to improving your love life. The point of this exercise is to get you used to talking to people you don’t know and form the habit of being more social.”
Matt Todd did a nice presentation on Rack to the Atlanta Ruby Group (ATLRUG) and they were nice enough to put video of the slides + audio of Matt’s narration online.
Aristotle Pagaltzis shows a pretty crazy technique for splitting a single git commit into multiple separate commits using an interactive rebase and successive checkouts on the same branch. Interesting approach. I usually pop the commit off into my working directory with git reset HEAD^ and then use multiple iterations of git add --patch + git commit until my working directory is clean again.
Quick presentation on Rack by Dan Webb. Covers a lot in eight minutes.
You cannot find this special offer anywhere else. A Twitter client, a fart noise generator, and a flashlight, all in one app for the iPhone. Act now!
Ian Bicking explains the connection between modern traffic planning and modern programming in dynamic languages.
I put a lot of work into this release. Really happy to see it out :)
Nice overview of caching from 1000 feet. Lays down some useful terminology, like “Cache Hit”, “Cache Miss”, “Storage Cost”, “Retrieval Cost”, “Invalidation”, “Replacement Policy”, etc.
Tom Preston-Werner shows you how to think.
True! A lot of cargo-cult types get this wrong.
Tanner Burson talks about one of the larger accomplishments of the Sinatra 0.9.0 release. We definitely need more docs on using Sinatra in this fashion.
Magnus Holm disects a couple of implementations for parsing nested form parameters (e.g., “person[name]=Joe&person[zip]=55555”) in Ruby. _why’s is the most interesting (as always). We just added this to Sinatra and I’m fairly confident we’ll see something like it land in Rack before 1.0.
Ruby based continuous integration server that rocks. Built on Sinatra and DataMapper. Painless setup, beautiful web UI, hooks up to GitHub. I wish I’d went and looked at this earlier.
The Prag’s have published two screencasts in a new series on Sinatra.
Guyon Morée shows how JavaScript 1.7/1.8 have been moving more and more toward Python with a few side-by-side examples. List comprehensions and generators would definitely be extremely cool to have in browser land.
A lost art, indeed.
“As I see it, we are all piles of slowly decaying meat, wrapped in the skins and hairs of other creatures, wearing colorful vestments made of skillfully woven plants, staring at what is essentially a pile of oil, metal, and sand all day. We spend most of our waking lives gathering piles of paper that we use to buy more dead animals, woven plants, and varying sized piles of wood, sand, oil, and metal. At night, we sleep in carefully crafted piles of these materials heated by barrels of decayed animals and plants and dream about having even larger stacks of paper with which we can use to impress other piles of sentient meat.”
Real artists ship.
The following screed was originally published at gist.github.com/54177 but has been copied here for posterity and because people keep asking me on IRC for the original link (it must be hard to find on gist for some reason).
We gave the Sinatra website a major face lift. Check it out. Don’t leave without subscribing to the feed.
I’ve written this same exact blog post a dozen times. For some reason, each hop along what should be a pure HTTP pipeline wants to invent their own psuedo-protocol for transferring HTTP messages. Why?! Your reimplementation of HTTP is not going to be any less complex — by definition, it must be at least as complex; and your reimplementation is definitely not going to be less buggy than the real HTTP implementations that have been around for a decade or more.
This is why can’t have nice things …
Christian Neukirchen’s utility for managing multiple virtual ruby installations.
Ruby 1.9.1 is out. Here’s the final rev of the NEWS file. Nice, condensed list of everything that’s new or changed.
Pat Nakajima’s script to measure the amount bloat you’re adding by requiring libraries. Generates a report on the number of methods added, along with a list of names. Interesting metric.
“Not to mention ‘overrideable’ is not a real word.”
“This module allows Ruby programs to access their OS’s native sendfile(2) system call from any IO object. Your kernel must export a recognized signature for the sendfile(2) system call to use this module. Currently, that includes Linux, Solaris and FreeBSD.”
If I was stranded on a desert island and could bring only one website …
This is one the amazing benefits of having an insanely simple but well defined SPEC (Rack) around the edges of your library. It makes it trivial to hook things up in new and interesting ways.
Aman Gupta and Joe Damato have implemented Fibers for the 1.8.6 and 1.8.7 MRIs. They’re patches now but will hopefully go into a 1.8.8 release. I had a chance to see these guys give a quick talk on their work at a local Ruby meetup — it was a hit.
I started full time with Heroku last Wednesday. This is why.
proggit on shoddy reporting by NPR.
Geoffrey Grosenbach interviewed me yesterday for the Ruby on Rails podcast. We had a nice chat about Python/WSGI, Rack, Sinatra, Rack::Cache, Heroku, and other random stuff.
Another interesting use of Rack middleware.
This has been sitting in the back of my brain for months now. I finally got a chance to throw something together last night:
Shotgun is an automatic reloading version of the rackup command that’s shipped with Rack. It can be used as an alternative to the complex reloading logic provided by web frameworks or in environments that don’t support application reloading.
The shotgun command starts one of Rack’s supported servers (e.g., mongrel, thin, webrick) and listens for requests but does not load any part of the actual application. Each time a request is received, it forks, loads the application in the child process, processes the request, and exits the child process. The result is clean, application-wide reloading of all source files and templates on each request.
“My mother used to say, ‘Butter is better with butter.’”
Big list of new features in bash 4.0.
Huge thanks to christian for getting this up. I’ve been meaning to get something on the Rack::Cache site for some time now.
This reddit comment makes me wish lesscode.org was still around :)
The RailsEnvy guys presented on a bunch of recent innovations in the Ruby/Rails community in their acts_as_conference 2009 talk. Go to 24:00 where Gregg gives a really tremendous overview of using Rack::Cache, the benefits of HTTP caching in general, and how to use all of this stuff in Rails 2.3.
I haven’t actually had a chance to watch this yet but I’m sure it’s great if it builds on the talk Gregg gave at acts_as_conference 2009. Also, I love this slide: “Reverse Proxy Caches – WTF?” :)
“I was having sort of a crap day, and spending 10 minutes watching this guy get towed completely brightened my day up. I normally feel slightly guilty watching everyday people’s cars get towed – so it was nice not to have any conscience eating at me this time.”
Harry Vangberg put together a Twitter relay bot in #sinatra (nick: nancie) so a bunch of the cool cats there are keeping the @sinatra twitter feed lit up with a stream links, tips, and announcements.
John Adams posted a bunch of details of the Varnish configuration they use in front of search.twitter.com to the varnish ML. Great stuff and nice to see the Twitter devs continuing to share their experiences with the community.
Why browser UI for HTTP auth is so horrible has always baffled me. This could be improved significant without any changes to HTTP whatsoever.
Things are starting to get interesting around here. James pulled together some (fucking sexy) high level architectural diagrams and annotated them just so. We can start talking about what we’re up to a bit more now that this is out. I’m jazzed.
I worked on this a bit. Jazzed to see it announced. Actually, they pretty much had everything working when I got there. I wrote some docs and tightened things up a bit is all.
Now go deploy something – it’s free!
Nice. This is very similar to the Sinatra::Test module but with a few additional features (i.e., the session/cookiejar thingy). If this gets traction (and it will), we’ll deprecate Sinatra::Test and recommend people use Rack::Test instead.
XPath-like syntax for expressing selection queries against JSON data structures. Interesting concept. I’ve always wondered why the basic concepts behind XPath were never borrow and applied to other types of structured data — it’s so insanely useful. I suppose jQuery popularized using CSS selectors for querying HTML but why not take the same basic concept and apply it to problem domains outside of SGML-inspired markup languages and their data models.
It’s important to understand how fork(2), pipe(2), and exec(2) work. I don’t want to hear anymore of this “fork is a hack” shit from any of you :)
For real. The best microwave is one that has a single timer dial that moves as time elapses. No one ever uses all those buttons. I use exactly one button on my current microwave: “Quick Minute” (hitting it as many times as needed) but I’d prefer a dial.
Amazingly painful to look at these. I hope motown can figure something out but I think it’s more likely that we’ll see other city’s (like Cleveland) follow in its footsteps. Decline sucks.
Aaron Swartz explains why he (and friends) put fixcnbc.com together. I’ve always been skeptical of petition sites but his logic is sound here.
Well said. It appears PHP’s culture of stupidity isn’t limited to technology. What a bunch of assholes.
Get it while it’s hot.
Good writeup on the rise of document and columnar databases, including Amazon SimpleDB, Apache CouchDB, Google App Engine, and Persevere.
OMG! Real, actual, research on the AIG bonus situation (from their 2007 10-K filing): “In light of the unrealized market valuation loss related to the AIGFP super senior credit default swap portfolio, to retain and motivate the affected AIGFP employees, a special incentive plan relating to 2007 was established. Under this plan, certain AIGFP employees were granted cash awards vesting over two years and payable in 2013.”
Interesting. So the bonuses were lined up in 2007 because all of their top guys would have left for other, less fucked up securities.
Mark Pilgrim: “Anyway, I now realize that there were some hidden assumptions behind my design decisions in 2000. Some of those assumptions turned out to be wrong, or at least not-completely-right. Sure, a lot of people downloaded dip, but it still pales in comparison to the number of visitors I got from search traffic. In 2000, I fretted about my ‘home page’ and my ‘navigation aids.’ Nobody cares about any of that anymore, and I have nine years of access logs to prove it.”
I don’t think most people realize how little site navigation matters anymore. Your site’s navigation is google, topic sites, blogs, and feeds. The “website” is dead. Long live the individual useful resource.
This is why simple is better. Sinatra probably runs well on any compatible ruby with a Rack handler.
Great ideas for tweaking Ruby’s GC after applying Stefan’s Kaes’s GC patch. By the way, that patch has been an option on the ruby port in FreeBSD for years. It works. Apply it.
Measure, Leading, Quotes, Rhythm, Widows, Emphasis, Scale, and Rags. Great piece.
We live in a crazy world.
Matthias Georgi’s framework for building DAV servers in Ruby with Rack. Could make building apps that mount into a local filesystem quite simple.
I had a chance to hang out with Rabbit’s Tony Garnock-Jones last week. Awesome guy. Knows his shit. We use RabbitMQ prettyy all over the place at Heroku — big fans.
Sick.
Protocols are hard. Nobody understands this.
Remi’s kick ass screencast on deploying to Heroku.
I’ve been staring at this screen for two sleepless weeks now. Really glad to have it wrapped. James Lindenbaum (CEO/founder/bad-ass) did most of the conceptual design work. seaofclouds did the fucking amazing illustrations and took the design to completion. Pedro Belo did the HTML/JavaScript and server side stuff. Definitely one of the best teams I’ve worked on.
We made it.
Nice. Probably more appropriate than tmm1-amqp in threaded/synchronous environments or when you don’t want to deal with EM. Then again, I believe tmm1-amqp has a synchronous interface. If not, it wouldn’t be hard to put one together with fibers.
James Carr’s classification system for unit test smells and anti-patterns. This is almost three years old but still extremely relevant judging by the test suites I’m working with today. I’m guilty of more than a few of these. Via @coda.
“Almost all non-functional programmers are unaware that tail calls facilitate a programming paradigm that they have never seen. The ability to tail call to functions that are not statically known is the foundation that makes many combinators useful. This is a style of programming where functions are composed in order to create a pipeline for values to flow through. Without tail call elimination, every stage in the pipeline would leak stack space and the whole approach becomes unusably unreliable.”
Jamie Zawinski (1997): “In this document, I describe what is, in my humble but correct opinion, the best known algorithm for threading messages (that is, grouping messages together in parent/child relationships based on which messages are replies to which others.) This is the threading algorithm that was used in Netscape Mail and News 2.0 and 3.0, and in Grendel.”
This will be my first talk at a major conference.
Kragen Javier Sitaker: “I have said that spreading false rumors in time of epidemic costs lives. People have asked me how…”
This is why I recommend everyone subscribe to kragen-tol.
Really interesting analogy between web architecture and a car crash. This is the piece that’s missing from almost every conversation about whether any given web framework or component “scales”. (via @jperkins)
Christian Neukirchen’s RailsConf 2009 wrap up. I had a pretty amazing time at the conference but sharing a Hookah with Chris was definitely a highlight.
Coda on why Rack has had so much success within the Ruby community and modeling projects after it in the future. I couldn’t agree more.
Here’s the slides from my RailsConf 2009 presentation on HTTP caching. I doubt the general info will make much sense without me talking over it but the diagrams should be fairly useful.
They shut the park down before Katrina and just left everything. There’s still prizes and stuff hanging on the walls. Spooky indeed. (Via @timbray)
Simon Willison is working on python web microframework based on Django. This will get interesting. Fast.
tl;dr — that’s why it’s awesome.
Nice. The xargs(1) switch -P N will run up to N separate processes in parallel. Combine with the -n M switch for a quick and dirty process pool.
Pretty. Y axis is a category of significance, X axis is the year. There’s at least one error: no mention of suck.com ;)
Tim Bray evaluates the web’s basic design from the perspective of the Fallacies of Distributed Computing. Reminds me of TimBL’s Axioms of Web architecture a bit. This stuff is essential to understanding why the web succeeded where other systems failed and why the web seems quirky in some ways compared to other distributed computing systems.
Minimalist, keyboard controlled (modal vim-like bindings, or with modifier keys) browser based on Webkit. A lightweight vimperator, maybe?
We’ve been getting a decent amount of PR-ish type coverage since the commercial launch but I still say blog posts like these are infinitely more interesting:
Remember when microwaves first hit the scene and people couldn’t believe how fast they could ‘deploy’ a meal? Yah me either, but the microwave changed the game big time.
And, unlike the microwave, Heroku doesn’t make your apps taste like cardboard :)
“If you have reached the age of 25, I have a bit of bad news for you, to wit: it is time, if you have not already done so, for you to emerge from your cocoon of post-adolescent dithering and self-absorption and join the rest of us in the world.”
Very well done.
Amazing! I put Ben under the table that night. Tucked him into bed and gave him a kiss.
Whoa. How do I get my hands on an english copy?
Great talk from this year’s gogaruca conference. Anything that starts with a rail against the belief that tools can have mystical scaling powers is going to end up being a good talk :)
Interesting reading if you found Nick’s handling of latency vs. throughput (in the gogaruca talk) intriguing.
Another classic on latency vs. throughput. This one gets into the limitations of speed of light fairly quickly :)
I want to believe!
mnot on how to evaluate different proxy cache options for your needs.
There’s a nice combination of old and new concepts in here.
The Boston Globe with a 30-40 good-res photos from the Iran election riots. It’s really poppin' off over there.
“… learn how to use httperf load testing with sessions, how to automate our httperf testing using autobench, how to graph the results from autobench, and lastly we talk briefly about a few other load testing tools you might want to be aware of.”
This is the second time Gregg has beat me to a great screencast :)
Tony’s simple HTTP interface to RabbitMQ. Somebody get this running as a service on EC2 so we can hook Heroku apps up to it on the private network.
Adam takes a look at how long requests and backlog interact. The sleep example runs concurrently under Mongrel but Thin and WEBrick will backlog.
Yahoo!’s proposal to open source their “fast, scalable and extensible HTTP/1.1 compliant caching proxy server” as an Apache project:
Traffic Server fills a need for a fast, extensible and scalable HTTP proxy and caching. We have a production proven piece of software that can deliver HTTP traffic at high rates, and can scale well on modern SMP hardware. We have benchmarked Traffic Server to handle in excess of 35,000 RPS on a single box. Traffic Server has a rich feature set, implementing most of HTTP/1.1 to the RFC specifications.
Rad. I know Yahoo! runs a custom build of Squid as well so I’m curious to understand where this thing came from. The proposal states that it was originally acquired from Inktomi and has been in use for some time.
Koz on Delayed::Job’s lesser-known send_later and handle_asynchronously methods. Sweeet.
Paul Graham at his finest. This is why I’m so pissy at the end of the day when I can’t get a single four hour chunk of time together. The thing is that you can do the same work in four contiguous hours what takes eight in interrupted hours.
Nice short list of PostgreSQL features everyone should be thinking about when choosing between MySQL and postgres (and each with a link to more detailed information). Then a wrap up that talks a bit about the licensing situation:
One of the major reasons for trying out PostgreSQL, regardless of it’s feature set, is it’s license and community. This can’t be more true these days, when MySQL AB was consumed by Sun Microsystems, which in turn recently got acquired by Oracle. While I doubt that the product itself may die off, the fact that the copyright to the code, and the direction of the database system itself may and will be dictated by a company like Oracle is a deal breaker for me.
I suppose it’s a bit on the FUD side but those are my feelings exactly.
This is an interesting idea. Publishing gems on rubyforge is complicated but the gem source is everywhere and gems have sane file names. Publishing gems on github is dead simple but the filenames include a username prefix, which is just kind of weird. Publishing gems to gemcutter is simple enough and the gems have sane filenames. This could work.
The Google Wave demo blew me away but I think Anil gets a lot right here. If the past is a good predictor of the future, Wave is a little too orphaned, a little too complex, and doing a little too much to be adopted quickly on any kind of large scale.
And every one is worth reading yet another “20 tips” post.
Sequel has become my ORM of choice for Sinatra apps backed by a SQL database. This is a quick Sinatra extension I threw together to remove some of the boilerplate required to get a new app setup. It includes a simple single-file migrations framework and some other stuff. See the README for examples.
I didn’t realize how much shit I had in my menu bar until I went through and did this:
There is a tried and true uncluttering trick for your home and office used by many professional and highly paid “Organizational Consultants”. It involves taking everything you own, save for the bare essentials, and moving it into storage. For a set period of time, retrieve items from the store when needed. Anything not accessed after said set period of time you likely do not really need and should be disposed of.
I have no idea why I ever had Sync (don’t use), TimeMachine (don’t use), Growl (useless), or Bluetooth (useless) enabled.
Safari extension that add’s a bunch of features I’ve missed since switching from Firefox: an awesome full screen mode, live search completion + results, open tabs to the right of current tab (instead of on the very very right), and a bunch of other stuff I won’t use.
Via Minimal Mac, my new favorite website.
It was terrifying to think that they may have actually considered making the page more edgy in some retarded acknowledgement of bling.com. Best redesign ever.
A discussion between Kirk McKusick and Sean Quinlan about the origin and evolution of the google file system (GFS). Really good interview. They’re fairly critical of the original design and how GFS is being used today. There’s also some discussion of an entirely new and incompatible version of GFS, designed from the ground up for the types of workloads GFS is being used for today.
New flight delay prediction site that actually works. Launched with an iPhone app. Runs on Heroku. I’ve been working with them for a few months now. Great team. Simple, smart product. More of that please.
Aaron Swartz with a classic mindfuck. Reader beware: this same basic set of stories was told to me by a friend a few years ago and bothers me to this day.
This seems to be a trend in Python and Ruby web circles. Frameworks present developers with a choice: accept these constraints, give up a little control, and I’ll make you more productive. Longer-lived apps start to evolve out of the framework at some point, though. You need more control over a piece here, or aren’t satisfied with the way something works there, and so you refactor and pull stuff out until the framework begins to slowly fade away. Maturity is a part of the web app development lifecycle we have precious little data on.
This comment nails it:
Maybe it worked out exactly like it should have. Django bootstrapped your app to a certain point. Got you further faster than you would have if you implemented everything from scratch. Then from there, you identified the things you considered inadequate and replaced them. If it all goes away who cares. You have learned something, shared it with us and moved on.
I think that’s just fine. Unexpected maybe, but fine.
I’ll admit, I’ve been holding out, hoping that _why’s disappearance was just a big misunderstanding. But at this point, I don’t think he could come back even if it were a big misunderstanding. Here’s a mirror (thanks mislav + GitHub Pages) of the Poignant Guide – the canonical reference for how to be. It’s our GOF; our SICP. I don’t think I’ve ever been so in awe of a piece of technical literature. Now it just makes me sad.
My God the typography is stunning. There’s currently only a single chapter — on Canvas — but I imagine the others will fill in rather quickly.
Big giant list of articles, essays, tutorials, and tech talks on making the web faster. This is part of a larger Google Code project that asks, “what would be possible if browsing the web was as fast as turning the pages of a magazine?”
Amazing, the way these things happen:
… in August 1969, Ken Thompson’s wife took their new baby to see relatives on the West Coast. She was due to be gone for a month and Thompson decided to use his time constructively – by writing the core of what became Unix. He allocated one week each to the four core components of operating system, shell, editor and assembler.
Constraints are powerful things.
Jonathan Dahl talks about minimalism and clarity in writing and how to use these principles in programming. Great talk. He uses a lot of analogy and audio/visual stuff at the beginning, which (in my experience) typically means that the presenter has no fucking idea what they’re talking about. Not here. Excellent, well-rounded talk.
Only requires that gdb be available on the box. No requiring libraries or listening on sockets to get the console. Rad.
Interesting way of breaking down various aspects of the World’s population. e.g., “If the World were 100 people, 1 would have a college education, 1 would own a computer, 1 would be dying of starvation.”
Interesting bookmarklet from arc90. Removes all superfluous content and administrative debris from the current page and turns the main content into something very readable. Comes with a few options for font-size and margin width. I’m rockin the Newspaper style with large text and medium margins. The entire web looks like my blog :) Love it.
Oh, and a tip for Safari + Webkit users: drag the bookmarklet to the first position in your bookmarks toolbar and you can use “Command + 1” as a hotkey.
Kudos to everyone hacking on Rails 3. This is a great demonstration of how far they’ve come in making it possible to pull in only the bits you need, without gem hell and activesupport spewing all over everything. Also, if you have an opportunity to see Yehuda’s “shit that’s happening in Rails 3” presentation, take it.
John Barnette with a must read piece for anyone that’s ever written a Rakefile.
I didn’t know about this:
In Snow Leopard, Apple has introduced a C language extension called “blocks.” Blocks add closures and anonymous functions to C and the C-derived languages C++, Objective-C, and Objective C++.
They go on to list code samples in each language. The syntax is … not what I expected. Check out the section on LLVM and Clang also.
Joe Damato and Aman Gupta show why and how Ruby (MRI)’s thread implementation sucks. Great presentation with lots of useful examples of using tools like strace and gdb to figure out where a process is spending time. Warning: you will be depressed and embarrassed after reading this if you currently target Ruby MRI.
Unicorn is a newish Rack-based HTTP server that’s kinda sorta like Mongrel but comes packed with some insane process management features. The main link is to the SIGNALS file, which documents the master/worker process model, supported signals, process replacement, failover, etc. See the README for a high level description of features.
This link brought to you by @defunkt, who explained Unicorn’s unique approach (repeatedly) over the course of a week.
Matt Todd and Mike Perham show off some nifty Ruby heredoc fu. It’s a gist, so fork it and add your own craziness.
Nice diagram. I’m way too familiar with this process.
Jacob Kaplan-Moss somehow pulls a bunch of interesting contemporary web development issues into a coherent essay. This bit on what happens when web apps begin to mature is especially well stated:
This is an impossible situation for framework developers: by optimizing for a quick start, by focusing on common needs, we’re essentially guaranteeing future failure. Remember the “Rails doesn’t scale” pseudo-controversy last year? I guarantee it’s only a matter of time until there’s an angry “Django FAIL” moment.
Frameworks ought to gracefully fade away as you replace them, bit by bit, with domain-specific code. (This is what I meant, above, that inter-op is also a scaling issue.) Right now, they don’t.
I wish more people would write about their experiences growing out of the general purpose web framework. It’s a totally natural thing but most people seem hesitant to talk about it because it’s interpreted as an attack on the framework or community.
I don’t even know how to summarize this. Let’s do this: follow this link if one or more of the following apply to you. 1.) you’ve read Godel Escher Bach and appreciated the anthill discussion between Achilles and Anteater, 2.) are interested in how brains work, 3.) like cities but not overly planned ones, 4.) would like to see evidence of a city constructed in the shape of an airplane.
Originally published in The Psychological Review, 1956, vol. 63, pp. 81-97. Every once in a while the internet decides to be useful and floats something old and amazing around anew. This is one of those things.
Mark has published the second chapter in his upcoming book, Dive Into HTML 5. I still can’t get past the typography. It’s so pretty.
Interesting W3C Note from January 2003 that I don’t remember ever seeing:
HTTP and URIs are the basis of the World Wide Web, yet they are often misunderstood, and their implementations and uses are sometimes incomplete or incorrect. This document tries to improve this situation by providing a set of good practices to improve implementations of HTTP and related standards (Web servers, server-side Web engines), as well as their use.
The information here is relevant to people who build web apps, not HTTP server implementors — the title is a bit misleading (not actually but practically). I especially like this bit about why short, less meaningful URLs are better than verbose, descriptive URLs. Shortness has become the most important characteristic of URL design in most apps I’ve built recently; SEO be damned.
Prime number browser and generator for all primes smaller than 10,000,000,000 (so called “small” primes). Enter a number and find the prime that follows it — super useful in a variety of server administration and manual cryptography tasks. Also, “there’s totally 455,042,511 prime numbers.” Neat.
Ian Bicking’s talk from DjangoCon 2009. Stimulating. I’m sure a nice comment thread will develop here over the next few days as well.
Dustin Sallings proofs out an implementation of the recently released Tornado web framework but builds on top of Twisted. The result is -1,297 fewer lines and all the benefits of having the Twisted framework underneath. I’ve been waiting for someone from the Ruby community to announce a port — we’re good at stealing. Using Dustin’s fork as a reference and basing a Ruby implementation on EventMachine might be the way to go.
Surprisingly interesting Esquire essay by Tom Chiarella:
Yes suggests pleasure. It wants something. Salesmen train themselves to use yes at the beginning of a sentence, no matter what, which is why when you say it enough, the word yes starts to feel like a con.
But no is cold and heavy. It puts an end to things. In that way, it is a word of control. Its very use suggests a speaker who actually knows something, who won’t bend, who won’t give in to what you want simply because you want it. No says the case has not been made.
Cops use it. Operators use it. Good teachers, too. I’d always wanted to be a guy who simply said no. So that’s what I did for a month. Whenever I didn’t want to do something, I didn’t hesitate, didn’t explain. I just said no.
“No.” Is there a more elegant sentence in the English language?
There’s a web that’s well-considered and worth savoring. We’ll show you where. — I fell in love with the term “Slow Web” immediately after reading that description, and the blog isn’t half bad either. If you’re a tl;dr type, or prefer to not wander outside the things on the web that are merely a collection of trivia, narrow, shallow, and sensational, then keep moving.
Via: Chris’s Trivium 13sep2009
Chris Wanstrath and Leah Culver’s submission for Rails Rumble ‘09 finally has its own permanent hostname. Hurl makes HTTP requests and then shows you stuff about the response, like headers and a syntax highlighted body. Hurl’s have permalinks, too, so you can link to them from email threads, IRC, technical documentation, etc. See the about page for more info and a screencast.
Whoa. There’s some serious shit poppin' off on the rest-discuss mailing list lately. Here’s Roy Fielding (completely out of context):
Quite frankly, this is the single dumbest attempt at one-sided “standardization” of anti-REST architecture that I have ever seen. It even manages to one-up the previous all-time-idiocy of IBM when they renamed their CORBA toolkit “Web Services” in a deliberate attempt to confuse customers into thinking they had something to do with the Web.
It doesn’t get any better from there :) I saw the REST-* site a few weeks ago but I (literally) thought it was a joke site. The sad thing is that, if the past is a predictor of the future, Jboss/Redhat will probably be able to convince a large chunk of enterprise IT managers that they are REST.
This is how I am using Rack::Cache, Sinatra, and CouchDB … Sweet ascii diagram there. I’ve seen this ETag chaining technique twice just this week. The other one is gemcutter. They store gems in S3 and pass the S3 provided ETag along in their responses, so it’s like the web app is more of an intermediary sometimes. Weird and cool and interesting.
It’s a rough world out there, and we need to to a better job of thinking about and testing under realistic network conditions. A better mental model of bandwidth should include:
- packets-per-second
- packet latency
- upstream vs downstream
Densely informational piece. Don’t miss the part where they generate packet loss using a microwave and a cup of tea :)
Chuck presents the results of a few ab runs against Thin and Unicorn. Surprising, even if the benchmarks lack rigor. I’d like to see good autobench runs for all of the backends on same hardware/network.
Eric Wong’s mostly pure-Ruby HTTP backend, Unicorn, is an inspiration. I’ve studied this file for a couple of days now and it’s undoubtedly one of the best, most densely packed examples of Unix programming in Ruby I’ve come across.
Unicorn is basically Mongrel (including the fast Ragel/C HTTP parser), minus the threads, and with teh Unix turned up to 11. That means processes. And all the tricks and idioms required to use them reliably.
We’re going to get into how Unicorn uses the OS kernel to balance
connections between backend processes using a shared socket,
fork(2), and accept(2) — the basic Unix prefork model in
100% pure Ruby.
But first …
Jacob Kaplan-Moss does the prefork echo server example from my Unicorn is Unix piece in Python. Awesome. Let’s see some more of these. Where you at, Perl?
Warning: PDF. This is probably the best high-level, everything about HTTP caching all in one place resource on the web at this point. Good stuff. I’m kicking myself for not being a part of his track at JAOO now.
Unix Network Programming, volumes 1 and 2, and Advanced Programming in the UNIX Environment. Yes.
Aristotle Pagaltzis comes through with the simple preforking echo server in Perl.
@paulsmith’s simple preforking echo server in C.
The CodeRack Rack middleware competition has begun. All entries get a $30 credit from Heroku and the top three pieces of middleware get special prizes (to be announced). You should submit something. I want to see it :)
GitHub documents the how and why behind their Unicorn setup. Also, for the record, the fork(2) + shared accept socket technique described in my Unicorn is Unix piece was first explained to me by Chris and Tom.
Jeremy Zawodny takes a look at the * is Unix thing and throws in some additional goodness: more on fork(2), the benefits of copy-on-write, and atomic file operations.
Brand new PeepCode screencast on Sinatra by Dan Benjamin. The production on this thing is really exceptional and they get into some meaty topics. Highly recommended.
Nick Quaranto shows why you need to start thinking about bundler.
“Frequently Questioned Answers,” that is. A big list of why C++ sucks, basically. Also, here’s some solid C++ bashing by notable programmers. All of this and I’m only half-way through Eli Bendersky’s recent essay on C++ hate.
Maintenance release that fixes a bunch of issues under Ruby 1.9, some multipart form problems, and various other minor bugs.
Matt Taibbi in his latest Rolling Stone piece on the banking fiasco: “We’re in a place we haven’t been since the Depression: Our economy is so completely fucked, the rich are running out of things to steal.”
It’s with great pleasure that I today announce:

I’ve taken a full-time engineering position with GitHub! It starts today.
I usually don’t like these web based regular expression editors but this one’s just right. Best part is that you can create a permalink for regular expression + text combos.
Redis’s SORT does a lot more than just sort stuff. Something clicked when I first saw this. I always wondered if/how more complex relational operations could be accomplished with key/value stores.
Simon Willison on using Redis with the Python REPL to get complex shit done quick. Insightful piece, as always.
Turns out
tig, the ncurses front-end to git,
is all kinds of customizable. Quick tip: set the author-width
to 5 or less and author names are abbreviated to initials:
$ cat <<E > ~/.tigrc
set show-date = no
set author-width=3
E
$ tig
RT harmful specs - disable them
KN [defunkt/master] I shine the shoes around here
CW don't let daemon serving ruin the public / private flow
CW ensure the listeners are notified, just in case
CW add "open sourced" event
CW Turns out we're still stuck on 1.0
CW let's try this again
CW allow overriding of the queue during job creation
Boom.
Cloud computing at its best.
Good article. The comments are even better:
reading this article reminds me countless time I have looked at the clock and the second hand it not moving and then it starts, I am sure this takes longer than a second
That’s always freaked me out. I’ve mentioned it once or twice but people just think I’m crazy so I don’t bring it up anymore. This guy, Frank, explains the phenomenon:
In more ‘joe sixpack’-terms. After you move your eyes fast, they are unable to collect information for a fraction of a second. When the eyes comes back ‘online’ the brain collects motion-information for an equal fraction of a second, and extrapolates the information backwards to create what things should have looked like and fills this fabricated visual information into your memory. Since the needle (or digit) didn’t move while the brain was collecting info for the extrapolation, it won’t be able to predict that it moved in the past either.
Crazy. I knew something was going on.
This is pretty rad. You can do web searches and whatnot without leaving a command line style interface but you can also do stuff like read news feeds. Check it:

I’d love to settle into this kind of workflow but these shell interfaces always have one thing or another wrong with them. Maybe this is The One. We’ll see.
Gemcutter will become rubygems.org and Rubyforge will eventually go away entirely:
So, what does this mean for RubyForge? The Ruby-specific functionality and data will be moved into RubyGems.org, and the parts that other hosting sites (GitHub, Google Code, SourceForge) can do better will be pruned away. Migration paths for those projects will be provided, we’re not throwing any switches without warning. RubyGems.org will not be gaining any “bloat” from rewritten RubyForge features.
Wow. Congrats to @qrush and the Rubyforge team for pulling this off so quickly. Also, it’s pretty cool to see Heroku hosting a major/official piece of Ruby community infrastructure.
It’s definitely misconfigured in one way or the other as evidenced by the fact that it sucks and has sucked consistently for over a year.
Nice little self-contained perl script that shows a basic memcached top display for a list of servers.
$ curl http://memcache-top.googlecode.com/files/memcache-top-v0.6 >
~/bin/memcache-top
$ chmod +x ~/bin/memcache-top
$ memcache-top --sleep 1 --instances memcache1,memcache2,memcache3
That gives you this:

Nifty.
I sometimes forget how useful a debugger is for coming up to speed on a new codebase. This bare-bones HOWTO on rdebug was everything I needed and nothing I didn’t.
Need more rdebug?
In honor of the greatest xkcd ever, this is Primer in its entirety. It really is that fucked up.
Not sure how I never heard of this program before:
dtachis a tiny program that emulates the detach feature of screen, allowing you to run a program in an environment that is protected from the controlling terminal and attach to it later.dtachdoes not keep track of the contents of the screen, and thus works best with programs that know how to redraw themselves.dtachdoes not, however, have the other features of screen, such as its support of multiple terminals or its terminal emulation support. This makes dtach extremely tiny compared to screen, making it more easily audited for bugs and security holes, and also allows it to fit in environments where space is limited, such as on rescue disks.
GitHub has rake tasks that use dtach to manage redis and maybe some other things.
I’ve dumped MacPorts for pkgsrc. This quick tutorial helped me get going and this package browser is awesome.
Really excited to see Chris release the shiny new Redis-based work queue that’s been running GitHub for the past couple of months:
It boils down to this: GitHub is a warzone. We are constantly overloaded and rely very, very heavily on our queue. If it’s backed up, we need to know why. We need to know if we can fix it. We need workers to not get stuck and we need to know when they are stuck. We need to see what the queue is doing. We need to see what jobs have failed. We need stats: how long are workers living, how many jobs are they processing, how many jobs have been processed total, how many errors have there been, are errors being repeated, did a deploy introduce a new one?
I’m still getting my feet wet with the jobs system but I can’t wait to get my hands dirty in the guts of this thing.
From the apparently just published, Coders at Work (Apress, 2009), Brad Fitzpatrick Talks About Programming:
In practice, nothing works. There are all these beautiful abstractions that are backed by shit. The implementations of libraries that look like they could be beautiful are shit. And so if you’re the one responsible for the cost of buying servers, or reliability – if you’re on call for pages – it helps to actually know what’s going on under the covers and not trust everyone else’s library, and code, and interfaces. (…)
They should have titled the book, “In Practice, Nothing Works”. Anyway, you can grab the — ick — PDF ebook for $20 on Apress.
I recently started a repository for my dotfiles, shell environment, vim config, and utility scripts. As of right now, I’m about 25% through all of the stuff in my $HOME — it should all fill in shortly.
Jacob Kaplan-Moss:
It’s really tempting to use an auto-documentation tool like Javadoc or RDoc for reference material.
Don’t.
Auto-generated documentation is almost worthless. At best it’s a slightly improved version of simply browsing through the source, but most of the time it’s easier just to read the source than to navigate the bullshit that these autodoc tools produce. About the only thing auto-generated documentation is good for is filling printed pages when contracts dictate delivery of a certain number of pages of documentation. I feel a particularly deep form of rage every time I click on a “documentation” link and see auto-generated documentation.
Hate that shit.
You know what I want? Man pages. For everything. Wouldn’t it be cool if you didn’t have to write roff?
Tim Pease’s unit test runner and reporter is sexy and has some really interesting features I’ve never considered when running unit tests. Here’s a piece of the README:
To use the solo runner.
turn --solo -Ilib test/
This will run all tests in the test/
directory in a separate process.
Likewise for the cross runner.
turn --cross -Ilib test/
This will run every pairing of tests
in a separate process.
Now that’s interesting. I’d love to have something like GNU Make’s -j option when running unit tests.
I’ve released a bertrpc library for node.js. If you haven’t play with node yet, set aside a night and dig in. Hacking async server-side stuff in JavaScript is every bit as awesome as I’d hoped: easy to install, good docs, fast VM, clean and simple event idioms. I’m impressed.
Chromium runs GreaseMonkey user scripts, apparently. It also says here that “script edits are picked up [from the file system] automatically; just refresh the page to see the changes,”
which is something that annoyed me with Firefox’s GM Addon — the files were buried, crazily named, and you had to use that janky addon manager bullshit to make/reload changes. What is that? A directory full of user scripts (now ~/.js on my system) is a perfectly simple setup. Combine that with right-now reloading and I might actually be able to develop these things when I need them.
Anyway, this AdSweep thing is what led me to investigate; looks pretty reliable if you can get through the GM setup.
I agree. I played with it for a couple days and now I can’t shut up about it. That’s basically all I talked about at RubyConf :)
This email exchange is quite possibly the funniest thing I’ve ever read on the web. Here’s a tiny piece:
Actually, you were asking me to design a logotype which would have taken me a few hours and fifteen years experience. For free. With pie charts. Usually when people don’t ask me to design them a logo, pie charts or website, I, in return, do not ask them to paint my apartment, drive me to the airport, represent me in court or whatever it is they do for a living. Unfortunately though, as your business model consists entirely of “Facebook is cool, I am going to make a website just like that”, this non exchange of free services has no foundation as you offer nothing of which I wont ask for.
It’s just pages and pages of that.
Thank you @mojombo.
UPDATE: @SimonEdhouse responds
UPDATE: zing
Perfect name.
ry’s talk from JSConf.eu. The leading paragraph says it all:
Node.js might be the most exciting single piece of software in the current JavaScript universe. Ryan received standing ovations for his talk and he really deserved it!
Wow. JavaScript is pretty damn big universe right now.
Nobody ever believes me when I tell them the Beastie Boys toured with Madonna in 1984-85. Here’s a great set of super high-res press clippings to prove it:
Why isn’t everybody this cool?:
Bill Adler was the director of publicity for Def Jam in the 80s. He allowed me to photocopy his press file on the Beastie Boys from the time when they opened for Madonna.
This is presently the topic of great discussion in the GitHub campfire.
Murray Gell-Mann’s (quarks, Nobel prize winner, peer to Feynman at Caltech) TEDtalk from 2007 on the relationship between beauty (simplicity) and truth in science and mathematics:
What is especially striking is that in fundamental physics a beautiful or elegant theory is more likely to be right than a theory that is inelegant.
He goes on to give a bunch of anecdotes that show this playing out over and over during the development of particle physics.
Tobi’s log tail and grep app is precisely what I’ve wanted on every single syslog machine I’ve ever had to deal with. And the code has some great examples of using EventMachine features to do real async HTTP stuff.
In comments related to the recent shutdown of Favrd:
I’ve spent the past year or so reading and writing and doing my level best to chip away at 40 years of belief in the logical fallacy that one’s identity meaning – self-worth, self-image, whatever you want to call it – can accurately be measured in the thoughts of others. Much as you and I may enjoy being encouraged through recognition and praise and dislike being saddened by rejection or indifference (god knows we’re taught to right from the outset by caregivers: good boy, pretty picture, heckuva job Brownie), deriving personal value from these transactions in the absence of a well-formed internal frame of reference through which you can decide on your own what does and doesn’t work, and subsequently accept the opinions of others as feedback, is just plain faulty thinking, of the sort that makes otherwise capable, centred people all loopy and weird.
Disco.
Google’s shipping official beta builds of Chrome for Mac and Linux. I’ve been using Chromium for a few months now and it’s definitely become my favorite browser. It needs a flash blocking extension and an ad blocker. I’m using userscripts for both but they’re a little janky.
I’ve released a tool for authoring UNIX manual pages using a markdown-ish source format:
Ron is a humane text format and toolchain for creating UNIX man pages, and things that appear as man pages from a distance. Use it to build and install standard UNIX roff man pages or to generate nicely formatted HTML manual pages for the web.
It still needs some work but can produce useful output for both roff and HTML. The sources are on GitHub.
defunkt’s hub is a command line utility that adds GitHub knowledge to git. Sweet. It expands GitHub repository references so you can do stuff like: git clone defunkt/gist, git remote add bmizerany, etc.
Nice to see Narwhal, Jack, CommonJS, and node.js getting some love on ReadWriteWeb. Javascript on the server is breaking out.
Tips for Ruby project maintainers on increasing the changes of getting your stuffs packaged for Debian. Most are just good sense. Use setup.rb, don’t explicitly require rubygems in your libraries and tests, use the most portable shebang (#!/usr/bin/env ruby), and provide a man page. Ron can help with that last one.