"Done! Now it works AND looks good" - Mads
"Just like the EchoDitto Dev team." - Dan
Tom Lee's blog
Don't Share Your Cookies: Drupal and the HttpOnly Flag
Jeff Atwood has an interesting discussion of the HttpOnly flag and how it can help protect your site from cross-site scripting attacks. For those hoping for a quicker rundown than Jeff's excellent explanation: XSS attacks are based upon the injection of third-party Javascript into your site. This usually happens via a comment or search form — really, it can be anything that accepts user input that is then rendered as part of the site's HTML. If that user input isn't properly sanitized it can trick other users' browsers into running the attacker's Javascript as if it were part of your site.
Once that happens the attacker has access to all sorts of goodies. The most valuable is the document.cookie object, which will generally contain the string that identifies your session to the site. They'll shuttle its contents off to their own server, and with your session in hand they can pretend to be you (at least until the session expires), doing whatever sorts of authenticated tasks you have access to on the site.
The HttpOnly flag tells most modern browsers that scripts shouldn't be allowed access to the cookie — it's for the webserver's eyes only. Handy! So what would it take to get this into Drupal?
Well, with the caveat that I haven't previously dug into Drupal's session-handling code, it looks like a relatively simple fix. In both d5 and d6 there's a function called sess_regenerate that's called at login and which sets the session cookie. This seems to be the only call to setcookie() in core, in fact, making me think that it's the whole enchilada as far as HttpOnly support goes. The line in question looks like this:
Capistrano and Drupal
I've been messing around with Capistrano over the last few days. The system is best known for deploying Rails applications, but at its heart it's really just a Ruby-based way of making it easier to manage repetitive server tasks over SSH.
There's one task in particular that I've used it to automate, and which you might find useful, too. Here at EchoDitto we're spread out over a number of offices — DC, Boston, New York, and often who-knows-where-else. This presents some development challenges when collaborating on sites built on a system that, like Drupal, puts alot of site configuration in the database, where version control systems can't be used to manage it.
Of course we do development locally when possible, but we also run a remote dev server to let us collaborate off a single database (and to allow clients to see our work). The obvious solution is to tunnel our local server's database traffic to the central server, allowing different developers' filesystems to connect to the same datastore. But this is slooow.
So other options: setting up MySQL replication? Well, that seems like overkill, and precludes offline development. The solution I've come up with is automating a simple database dump from the dev server. When I need to make config changes I do it on the dev site; when I'm working in the filesystem and need fresh data, I simply invoke a command to pull down the dev DB. For most sites this only takes a few seconds to run, and invoking it is as simple as running `cap get_db` from anywhere within the dev site directory structure.
I used to do this with a somewhat horrible bash script. But it made various assumptions about filesystem and had to be modified for each new local project. Capistrano has let me make this a bit more abstract and reusable. The following script still makes a few assumptions about our servers and configuration, but you might find it useful, too.
Drupalcon Szeged: What I'm Missing
Well, I'm not in Hungary, and it's a drag. Drupalcon has begun! It looks pretty exciting, but at EchoDitto I think we're collectively keeping our fingers cross for Drupalcon DC.
Still, I'm doing my best to review the stuff coming out of Szeged. So far it's just a few sets of slides, but I'm optimistic that we'll get some podcasts or maybe even Qik streaming before too long. Inspired by Alex's list, here are the sessions I'm bummed to be missing — not that I could have physically attended them all, of course. But these are the ones that look particularly interesting to me:
CSS Aggregation Problems in Drupal 5
This may be old hat to some of you, but it was a near-revelation to me (and tough to find through Googling) so I thought I'd share it and potentially save others some hair pulling.
If your Drupal 5 site goes crazy when you turn on CSS aggregation, try examining your stylesheet(s) for absolutely-addressed url()s. There was (and is) a bug in Drupal 5's CSS aggregator code — since aggregated CSS has to live in a different directory from the original stylesheets, all those URLs have to be rewritten. The function that rewrites them expects them all to be relative, though. So:
background-image: url(/sites/all/themes/zen/xxx/images/something.gif);
gets turned into:
background-image: url(/sites/all/themes/zen/xxx//sites/all/themes/zen/xxx/images/something.gif);
That's obviously no good. Instead, simply make sure that your URLs are all relatively-addressed:
background-image: url(./images/something.gif);
That should be rewritten properly by the aggregator.
The more complete solution, of course, is to fix drupal_build_css_cache(). There is a patch available, but since the issue has been more thoroughly addressed in Drupal 6 and 7, not much attention is being paid to fixing it in 5. And that's probably okay: in most cases it's going to be quicker and easier to simply edit your stylesheet to use relative paths. It's best to avoid patching core when possible, and this is no exception.
Proxy-based Mischief with OpenWRT
I hope that most of you have already seen the upside-downternet. This hack has been around for a while, but that doesn't make it any less brilliant: thanks to some Linux trickery, every user connected to your network gets served their images upside down. It's pretty clever.
It's also similar to a related but not-quite-identical application I've had in mind. So how does it work? The magic arrives in the form of transparent proxying, a technique whereby a router sends its traffic through a proxy server without the user responsible for that traffic noticing anything (or having to change their configuration). This is called "transparent proxying". In the case of the upside-downternet the Proxy server was Squid, an awesomely powerful, sprawling application. In this case Squid runs on its own machine; the router is configured to reroute its traffic through the Squid server before letting it out onto the net. The Squid server keeps an eye on the URLs being requested and uses a custom script to rewrite any that look like images. The replacement URLs point at a web-accessible script on the Squid server that knows to fetch the original image, flip it upside down, then serve it back.
As I said, I wanted to do approximately the same thing. But I wanted to do it all on the router. And I don't mean router in the systems engineer sense of "router". I don't mean a box sitting in a datacenter with fiber channel running into the back. I mean a box sitting on a shelf in Best Buy with a peel-off coupon on the front.
Firebug Inches Along
It's been too busy around these parts for me to put together a post about the exciting things happening in the DC tech community, particularly in the hobbyist electronics space (short version: they're exciting!). But I can at least share this tip.
Many of you are probably using the newly-released Firefox 3 release candidate, and it seems likely that some of you are running into the same problems with Firebug that I was, particularly when using its Javascript console. Big, angry errors with "NS" and "SECURITY" in them? Yup, that's the one.
Well, get thee over to the firebug downloads page and grab the latest alpha. It solves this problem, but introduces a new one for those debugging a jQuery-enabled page: the jQuery object is nowhere to be found.
Actually, it's not that bad: you just need to do something like this
$ = window.$;
Before you start mucking around with scripting from the console. It's irritating, but I'm sure it'll be fixed soon. And in the meantime, you could always whip up a Greasemonkey script to do this job automatically.
If you want to know what's wrong with our industry
...look no further than the comments on this TechCrunch post. Reacting to the scoop that Twitter is dumping Ruby on Rails (which almost immediately turned out to be wrong), more than 150 people decided to chime in about what Twitter's scaling problems are and how their own preferred web framework never would have encountered them. A whopping one person managed to refer to Twitter's custom message queueing software by name.
The rest presumably just like the (drag & drop HTML creation in Visual Studio|error detection in Zend|Google-enabled buzz surrounding Python), and have read "Rails doesn't scale" somewhere. I suppose their prattling is still slightly more meaningful than the yammerings of the web's legions of widget evangelists, social network triumphalists and self-proclaimed "SEO experts". But not by much.
At any rate, I think this is a good reminder of the signal:noise ratio facing our clients as they attempt to find technical help in a crowded marketplace. If you're not a technical person it's genuinely hard to tell the difference between someone who knows what they're talking about and someone who's simply regurgitating buzzwords. My rule of thumb when encountering bold pronouncements like the ones in that TC thread: ask "why?", then count the number of unexplained acronyms and buzzwords in the answer.
Fun With Piped Emails
One of the things that I've learned over my years at EchoDitto — that's been hammered home again and again — is the importance of email. Twitter's cool, social networking is great, but your online strategy absolutely has to account for email. It's not glamorous, but it's important.
The same is true on the tech side of things. It's easy to forget: on a day-to-day basis, my wranglings with email generally include dealing with spam blacklists, ensuring that scripts don't function as open relays, and writing templating systems that'll be used to send mail. All of it's pretty boring. But it's worth keeping in mind that email, when piped to a script, can serve as the infrastructure for some pretty neat services, too.
I did this two years ago for SXSW, building an SMS app on the cheap by counting on the mobile carriers' SMS-to-email functionality. It worked pretty well, although it was a rat's nest of Perl scripts.
This week I took a pass at another application in Ruby (with a PHP frontend), and this time the resulting code is a bit less cringeworthy. The idea is simple: set your Twitter "new follower" email notifications to go to a custom email address. They'll be piped to a script, disassembled and the new follower's statistics analyzed. If it looks like the new guy is a spammer or bot, they'll automatically be blocked. If not, they won't be. Either way you'll get an RSS notification about it. You can try it out here, if you'd like.
It's not exactly going to set the world on fire, particularly since Twitter is expected to release similar functionality soon. But the project does serve as a pretty good template for how a piped email service can work.
Son of Zipcode-to-Representative Database
Remember this guy? Yeah, I barely do, too. But way back in 2006 I put some of the lingering knowledge I had during an earlier job together with a newfound passion for writing horrible rats' nests of Perl code. The result was a set of scripts that could scrape the rather lame HTML interface to House Information Service's zipcode+9-to-congressional-database matching service.
I'm sure my scripts haven't aged all that well, but Aaron Swartz was kind enough to take up the torch and share a set of Python scripts he coded to accomplish the same thing. Thanks, Aaron!
Those of you looking for zipcode-to-congressional-district matching would do well to check those scripts out. You might also want to follow the threads from this tweet, in which exoDitto Tim Jones asks the LazyWeb for other solutions to the problem.
Dorkbot Tomorrow, Drupal on Wednesday
I'm going to be presenting at tomorrow's Dorkbot DC meeting — anyone who's interested should come on by. I'm going to be talking about making a Fonera router talk to an Arduino, a subject I first blogged about right here.
And of course the DC nerd community also shouldn't forget about Wednesday's Drupal Meetup at Affinity Lab, which is once again being hosted by our friend Mike McCaffrey. You can find details here. I think it's safe to say you'll be able to find some of us at each of these events.
Using Blip.tv With FeedAPI
Yesterday Development Seed was kind enough to give Chris & me a rundown of how the Drupal community is organizing its participation in the Google Summer of Code program. Along the way I got a chance to chat with Alex and Ian about FeedAPI and FeedAPI Mapper, two excellent projects that DevSeed ushered into being through last year's Summer of Code and now continues to maintain and extend.
I've just begun using FeedAPI for the first time in a project destined for production, and so far I'm very pleased with it. It offers a more fully-considered alternative to aggregator.module — and with the addition of the optional Mapper module it becomes simple to turn aggregated RSS items into Drupal nodes, with the items' attributes stuck in whatever CCK fields you care to create. It's really slick.
In my case I'm using it as an integration point for Blip.tv. Our client needs video capabilities, but I saw no reason why we should mess around with transcoding, customizing an FLV player and all the rest of the headaches that come with web video (been there, done that). Blip does all of that stuff very well, and has social features baked in, too. I'd rather just have the client upload their videos there, then count on FeedAPI to turn them into nodes that can be exposed through Views. Any configuration that we can't get from the Blip RSS feed can be manually handled by an editor — Workflow-NG fires off a "please come edit and publish me!" email whenever a new video node is created.
Drupal Data Importing as a Thrill-seeking Behavior
Yesterday Ethan, Chris and I found ourselves needing to move some data into Drupal — we're building an existing client a new site and they need some blog posts moved over from their old one. It sure sounds simple. But as anyone who's actually done data migration from one blogging platform to another (or just from one version to another) can attest, it's rarely that easy. There are a few options, though, which I present here in increasing order of their likelihood to unexpectedly spiral into a huge, awful mess:
Presence Detection via the iPhone and Wifi
So, as Phil mentioned, we all got iPhones as a holiday bonus. And they're pretty great. JP and I held off until Macworld, lest we miss out on a new 3G handset (a longshot, I know). When that didn't happen I immediately scurried over to Pentagon City and bought a beautiful internet lozenge.
Since then I've been figuring out all of the amazing things it can do for me. On Thursday I had a stroke of inspiration. I've been looking longingly at Bluetooth presence detection setups for a while. I like the idea of having my hardware respond to my proximity, but I've got some reservations about wasting a Bluetooth dongle (and server, and Bluetooth-on-Linux configuration time) on the effort. The iPhone is the first mobile I've owned that does wifi. This seems like an opportunity to do presence detection on the cheap.
My approach is pretty simple: ping the iPhone. If it's there, I probably am, too. But how to ping it? The iPhone uses DHCP to get an address. At home I've got a router running the SveaSoft firmware. It's simple enough to configure DNSMasq to always dole out the same IP address to my iPhone's MAC.
Then I wrote a bash script to send three pings, check the number of successful replies, and log the result to a text file. I set it up to run on a minute cron and let it go overnight:
PING_COUNT=`ping -c 3 192.168.2.21 | grep "bytes from" | wc -l` test $PING_COUNT -eq 3 && RESULT="here" || RESULT="not here" RIGHT_NOW=`date` echo "$RIGHT_NOW - $RESULT" >> status.txt
Here are the results from the first 24 hours:
Dorkbot DC - January 16, 2008
On Wednesday Ben and I headed over to the monthly meeting of Dorkbot DC and soldered our little hearts out. I've only been to one Dorkbot before (well, unless you count the one at SXSW — but that was mostly an opportunity to buy t-shirts and drink free beer). My previous experience had been enjoyable, but maybe a little dry. It's inevitable that the speakers won't pique everyone's interest every time, I suppose.
This meeting was a lot more fun, and a lot more hands-on. The organizers did a fantastic job, preparing instruction material, assembling kits and even pre-drilling jigs for the rest of us in an effort to introduce the extremely large crowd to soldering by way of MAKE Magazine's LED cube weekend project, slightly modified to work with an Arduino.
Sure, it's a simple project, but we've never claimed to be electronics gurus. Besides, it was a great opportunity to refine our soldering skills — something I'm in sore need of after nearly trashing my Wii during a botched modchip installation.
Bug Labs Announces Pricing (and it's not too bad!)
It looks like Bug Labs has set some prices for its open hardware initiative. And frankly, I'm pleasantly surprised. Yeah, $350 for the base unit isn't cheap, nor is the $100ish average price of the add-on modules.
But after reacquainting myself with this video, it all seems pretty reasonable. The base unit runs Linux, has a wired ethernet connection and storage capabilities. The add-on LCD is the same unit as ipods ship with, and has touchscreen capabilities thrown in. All in all it seems to be a very high-quality product. I don't think I'm going to rush out to buy one — I'd rather clog my breadboard with torn-out hair and call it a "learning experience", masochistic though it may be. But I certainly won't complain if Nicco shows up with one in the office.







