When we launch projects at Backcountry.com, one of the questions we ask ourselves is "what are the risks of launching this?" Sometimes it is tempting to say "this is a small project that doesn't touch any core functionality, so the risk is low." The problem is that our current stack is synchronous, has blocking IO and is fairly monolithic. The result of this is that most any problem can bring down the entire application. For example, if you create a query that is too slow, the Apache process that is executing that query is waiting a long time for that response. If you check your system stats, you'll see that CPU usage is low on your app servers, but you might see memory start to get eaten up. What? Memory is getting eaten up on the app servers that are just waiting around? Yup, that's right. Although your app servers aren't doing much, the threads or child process are building up because they can't complete while they are waiting for a response from the database. If you're running Apache prefork-MPM, you've got processes taking up around 1MB or more, one per request, just sitting around.

Now let's see how this looks in an evented, non-blocking I/O design (such as Node.js, but there are others too such as EventMachine). In Node.js, there is, for the most part, only one process and no threads (there are exceptions to that, but nothing that will affect this example). As client conenctions are made, they are added to the event loop, which is basically just a queue of stuff to do. When the database query get's executed, the code will say "hey, i'm executing a query, which might take a while, so please just throw me back in the event loop and call me again when the query is finished." So, what happens on the app server when the query takes a long time to execute? Well, that one request won't get a response for a while, but that's about it. Barely any memory is tied up and no processing time is wasted on waiting around.

Furthermore, in situations where you will have large amounts of socket connections to clients, because Node.js requires so little memory for each connection and handles idle connections so gracefully, it can handle a phenominal amount of concurrent connections. You can see some roungh numbers at Node.js Has a Bright Future, an article I wrote when I first discovered Node.js. Those examples were oriented toward normal HTTP connections. It would be even more interested to see how something like Faye (running on Node.js) would stack up against the proven CometD/Jetty.

Lean more at nodejs.org.