Lunch With David Yields a Link Crawler

I had tasty lunch with David at the China Restaurant yesterday. It was good to see an old friend that I met on the Internet ages ago. Hopefully sometime soon I'll get to see his babby.

Anyway, he mentioned that he was in need of a link crawler while at lunch yesterday. As we all know, I have trouble resisting psuedo-challenges like that... and oh look. What's this? Could it be? A garbage file? Put there by none other than Elingson's elite sysadmin... oh, what's this? Is this the secret bank account that the money was stashed in? Oh, I think it is. Man. You know, I kind of feel like God...

Oh, wait... sorry. Really and truly, it would appear as though rather than my usual night of BioShock and/or Metroid Prime: 3 I tore myself away to write some code. Well David, here's your link crawler! It's not heavily tested or terribly robust, but it runs against a handful of test sites and I think it does exactly what you described.

Update - 2007/10/02

I've published a bazaar repository for this project. You can find information and an updated version of this file at this posting.


david

woah.. thanks man! I will have to install ruby on our linux box here and give it a shot. I'll let you know how it fares. Thanks a ton!!! We'll have you guys over sometime...

Jordan T. Cox

:) Glad you like it! Let me know how it works out.


Required
For gravatar support
Required