One of the things that I've learned over my years at EchoDitto — that's been hammered home again and again — is the importance of email. Twitter's cool, social networking is great, but your online strategy absolutely has to account for email. It's not glamorous, but it's important.

The same is true on the tech side of things. It's easy to forget: on a day-to-day basis, my wranglings with email generally include dealing with spam blacklists, ensuring that scripts don't function as open relays, and writing templating systems that'll be used to send mail. All of it's pretty boring. But it's worth keeping in mind that email, when piped to a script, can serve as the infrastructure for some pretty neat services, too.

I did this two years ago for SXSW, building an SMS app on the cheap by counting on the mobile carriers' SMS-to-email functionality. It worked pretty well, although it was a rat's nest of Perl scripts.

This week I took a pass at another application in Ruby (with a PHP frontend), and this time the resulting code is a bit less cringeworthy. The idea is simple: set your Twitter "new follower" email notifications to go to a custom email address. They'll be piped to a script, disassembled and the new follower's statistics analyzed. If it looks like the new guy is a spammer or bot, they'll automatically be blocked. If not, they won't be. Either way you'll get an RSS notification about it. You can try it out here, if you'd like.

It's not exactly going to set the world on fire, particularly since Twitter is expected to release similar functionality soon. But the project does serve as a pretty good template for how a piped email service can work.

Sometimes I wonder why I ever bothered to compile wget for OS X. I keep finding clever new things that curl can do — even some that make me wonder why I bother with Perl and Ruby's Mechanize module/gem.

For instance: today I found myself needing to export a friend's Typepad blog. He's got a lot of content, and I wanted to pull it down on a speedy connection and then gzip it before pulling it onto my laptop over my sluggish DSL (Typepad unhelpfully delivers its exported content in an uncompressed non-ML format).

Typepad provides a URL for the export, but it only works when you've authenticated through their website. Curl makes working around this pretty simple, though, as I found out from this Ask Metafilter thread. Here's how it works:

curl -k -d "__mode=redir&next=http://www.typepad.com/t/app&username=user%27s+name&password=their%20password" -c cookie.txt https://www.typepad.com/t/app

You simply use -k to turn off SSL requirements, -d to pass a series of key/value form pairs (properly url-encoded) to the URL to which the form posts, and -c to output the returned cookie to a file.

Then you pass in the session-containing cookie file, allowing you to stay authenticated on subsequent requests:

curl -b cookie.txt -o complete-export.txt "http://www.typepad.com/t/app/weblog/post?__mode=export&blog_id=YOUR_BLOG_ID"

Naturally, you'll want to replace the URL in that last request with the one corresponding to your own export URL.

It all worked perfectly — well, up until the point at which I discovered that Typepad exports seem to inevitably fail once they top 100 megs. Seriously guys: file compression. Try it, you'll like it.