SonicChicken blog

I was posting the next meeting of the Sacramento Go Club on the Sacramento Boardgames, Cardgames, & Miniatures Meetup Group, but this week for some reason Meetup’s software is giving me grief about the event’s URL. I’m guessing it’s barfing on the ‘~’ character in the URL, which is perfectly legal (and worked fine until a few days ago). I really want a URL for the event, so I’m posting it here. Thanks, Meetup. It’s not enough that you’re charging me to organize cool events, now I’ve got to do fake redirects to keep your damn site happy.

Kevin Burton frets about a potential “ping crisis”:

…in the long run it seems to be a big difficult (sic): 1. Everyone wants to spam you. 2. You’re not a consumer level service so you can’t run Ads to make money. 3. The more traffic you get the more your costs go up. 4. Scaling a system is difficult once you get to that volume. 5. All your pings need to be delivered fast (see #4).

Those are some interesting points to ponder. Point #1 is particularly troublesome, so let’s think about points 2-5 first. It seems to me that we’d do well to look at some other services for ideas and inspiration. The first one that comes to my mind is DNS. No ads on DNS, and not many fees are being charged for it. Perhaps we should come up with some sort of distributed model for handling blog pings? If we have a distributed service, no single server would have to bear the entire load, and we can probably avoid the traffic cost and scaling issues. But now about point #5 — Obviously pings need to be acknowledged very quickly, but do pings need to be delivered fast? Joseph’s ping service is based on a queueing model, which seems reasonable and more practical to me. What do I care if my posts are advertised “immediately” or I have to wait 5 minutes?

So now back to point #1, blocking spam. Do we need some sort of RBL? Open-source tools to identify spam? I really don’t know the answer to this. We need some input from people on the front lines about this topic.

Ping-o-Matic (the default pub-notification service in WordPress) is a great idea: You publish a new post, and WordPress notifies PoM, and PoM passes the update along to a whole bunch of update services. Very slick! But there are two problems with it:

1. Single point of failure. If PoM is down (or busy or otherwise unavailable) when you click the Publish key, you’ll get to watch your browser spin in circles for a bit while it tries to contact PoM.
2. Real-time pinging. If PoM is available when you Publish, it seems like it holds you connection while it notifies all the update services.

Joseph started discussing these problems with me the other day, and the delays I’ve sometimes noticed upon clicking the Publish button came to mind. Turns out that he had an idea for a streamlined notification service, which he’s now opened up for testing. I’ve installed the new service in place of PoM, and it seems to be a significant improvement. I’d like to collect some data comparing the new service to PoM, to get a more objective look at this.

Some things to note:

• Joseph’s service could still be a single point of failure, assuming one were to replace PoM with this service in the Options/Writing/Update Services setting.
• The basic idea of the new service is to respond to a blog’s update notification as fast as possible, then later to notify all the update services. The cost of this approach is a small delay (a few minutes) between publication and notification, but the quick response between clicking Publish and getting back to my blog admin page is worth it!

My buddy Andy just launched a hilarious evacuation plan store on CafePress. I picked up a couple of coffee cups (large, of course). Hopefully they arrive this week — I can’t wait to get a look at the actual merchandise!

This whole “Aim For Tree” thing started about two years ago, when evacuation plans were being posted in our old building. We saw this nice little frame on the wall, which looked like it was designed to hold a piece of paper or something. The frame had “Evacuation Plan” printed on it, but there was nothing inside the frame. We were on the fifth (top) floor of our building, so Andy drew a 5-story building with a stick figure jumping out of the window and into a tree. We slid this into the Evacuaion Plan sign, figuring that it was better than nothing! Now you can get your very own “Aim for tree” merchandise at cafepress.com/aim4tree/!

Update 2005-11-10 20:55:29 UTC: Posts concerning my Powerbook boot failure are tagged with powerbook and boot_failure. Posts about my experiences with DiskWarrior are tagged diskwarrior.

Nooooo! My trusty PowerBook won’t boot this morning. The screen lights up, the Apple logo appears, and the spinner starts spinning — and it hasn’t stopped after more than an hour. A little googling turned up a post on how to boot a PowerBook in single-user mode, so I held down Command-S while powering on. That worked, cool! Then I tried /sbin/fsck -y. It turns out that my disk has journaling enabled (I didn’t know that), so I had to use the “force” option, /sbin/fsck -f, which gives Bad News:

localhost:/ root# sbin/fsck -f
** /dev/rdisk0s9
** Root file system
** Checking HFS Plus volume.
** Checking Extents Overflow file.
** Checking Catalog file.
   Invalid key length
(4, 4341)
** Volume check failed.

So, time to try the Disk Utility from the install CD — hold down the Option key while powering up. This gives me three options: boot from local disk (which I already know doesn’t work); boot the install CD (which I’ll try in a minute); and Apple Hardware Test, which sounds like something I should try right now.

Apple Hardware Test (PowerBook version 2.0.6) loaded and I selected the Extended Test. After about 3.5 minutes, the utility reported that the Mass Storage system had passed hardware test, so it looks like my hard drive is okay, although the data on it appears to be corrupted. I let the rest of the hardware test run, just for kicks. My PowerBook has 1 GiB of memory, which took about 26 minutes to test. Total test time was 33.5 minutes, with everything passing. (phew!)

I rebooted holding down the Option key again, this time booting from the installer CD which took about a minute to load. When the “Install Mac OS X” window opens, select the Installer menu then Open Disk Utility... item. In the Disk Utility, I selected the disk device (vs “Macintosh HD”) and clicked Verify Disk on the First Aid tab. It reported:

Verifying volume “Macintosh HD”
Checking HFS Plus volume.
Checking Extents Overflow file.
Checking Catalog file.
{red}Invalid key length
The volume Macintosh HD needs to be repaired.


Volume check failed.

Error: The underlying task reported failure on exit (-9972){/red}


1 volume checked
  {green}0 HFS volumes verified{/green}
  {red}1 volume failed verification{/red}

Repairing disk for “Macintosh HD”
Mounting Disk(S,”Checking HFS Plus volume.”,0)
Checking Extents Overflow file.
Checking Catalog file.
{red}Invalid key length
Volume check failed.


Error: The underlying task reported failure on exit (-9972){/red}


Repair attempted on 1 volume
  {green}0 HFS volumes repaired{/green}
  {red}1 volume could not be repaired{/red}

This post suggests using Disk Warrior. I can’t use a download version, since I need a bootable CD to repair the non-bootable drive in my laptop, so I’m off to CompUSA to pick up a physical copy.

Back from CompUSA, with my wallet $100 lighter, ouch! This had better work… I booted the DW version 3.0.3 CD (hold down the C key while powering on and insert the CD). The single-sheet instructions indicate that it will take 5-15 minutes to boot. In my case, it took about 6 minutes. After a few prompts, the software got to work. It got through the first few steps quickly, but then started “Step 5″, something about locating directory information. A message appeared on the progress window, “(Speed inhibited by disk malfunction)”. I assume this means file structure corruption, as opposed to a hardware malfunction of the actual disk. Well, the speed was definitely inhibited! It took 14.75 hours to complete “Step 5″ on a 75 GiB disk which was approximately half full. The worst part about this was a complete lack of any sort of progress indicator. The only way I could tell that anything was happening was to put my ear right on the laptop case over the disk, so I could hear the disk chattering once in a while. (Fortunately it wasn’t the cyclic series of sounds that a failing disk makes; instead, it was the purposeful sound of a disk doing random seeks. If you’ve listened to disks, you know what I’m talking about.)

After the excruciatingly slow “Step 5″, the next step, “Constructing optimized replacement directory…”, went relatively quickly. I didn’t keep track of its elapsed time, since I was trying to get ready to go to work and I missed the transition from step 6 to 7. In fact, steps 6, 7, and 8 all completed too quickly for me to note their elapsed time, but they took about an hour total. I started to hope that maybe I might have my PowerBook operational very soon, but then “Step 9: Recording any file or folder differences…” started. This took about 3 hours, but at least this step had a progress bar!

DiskWarrior has finally finished, after about 19 hours. After reviewing the log file and the new directory structure, I accepted the changes (as if I really had a choice). It took about 40 minutes to update the disk, then reported: DiskWarrior encountered an unexpected error while attempting to show the disk “Macintosh HD” on the desktop. Rebuild this disk again. Preview the disk and backup all files from the preview disk (2168, 4203). ARRRGHHH!!! After doing some post-DW diagnostics, it appears that the disk was hanging by a thread, and the heavy disk activity caused by DW was too much for it. The system will no longer boot into single-user mode.

This post is getting too long already, so I’ll publish it in it’s current “unresolved” state. I haven’t given up yet, though…