The Treehouse Blog

A year or so ago, I lost my BT878-based 4-port video capture card, apparently to lightning. The card was connected to a cheap black-and-white camera outside my house, intended for security monitoring purposes. When the card failed, it stopped showing up in ‘lspci’ and the box was otherwise fine, so I didn’t bother taking out the card. That box was decommissioned about 6 months ago, and I finally took it apart this week, revealing this:

Oak had an upgrade this week. It’s been running for years with an AMD Athlon XP 2400 on an Asus A7V8X motherboard. With the new tasks of running a 5-disk software RAID5 array and H.264 video codecs added over the past few years, it has started lagging behind. The processor and motherboard have been upgraded to AMD Athlon 64 X2 5400+ Brisbane running at 2.8GHz and 65W, and a Gigabyte GA-M57SLI-S4 motherboard. That should provide something around 2.8x the processing power, more disk I/O capability since the motherboard provides 6 onboard SATA ports instead of the two PCI-to-SATA cards I had been using which would max out at 133MB/s, and I’ve also quadruped my RAM from 1GB to 4GB.

Instead of upgrading the OS in place, I opted to do a fresh install of Fedora 9 x86_64 onto a software RAID mirror, replacing the old 10-gig non-UDMA IDE system drive that I had been using. This was relatively painless. A fresh mythtv install combined with importing my old database resulted in a working system. I had to copy over my lirc customizations, of course. The on-board sound with the default PulseAudio configuration works fine for everything including the raw digital output for AC3. One quirk I may need to look into is that when skipping around in a AC3 video, there will be an absence of sound for a considerable portion of a second before it comes back. This quirk is better than the problems I’ve had recently with my (ancient) SoundBlaster Live which would cause the receiver to not switch back to PCM mode on its own after an AC3 stream stopped.

I also upgraded to a GeForce 7300GS-based PCIe graphics card which has allowed me to switch to ‘gl’ rendering for mplayer, which seems to fix most or all of my frame tearing issues that I would experience when using 1024×768 to drive the TV output.

Still some issues to resolve, and I haven’t measured it’s power draw yet, but the upgrade process has been a lot less painful than I had feared.

An altered detention letter brought my alma mater into the international spotlight (on the web, at least) today. An unnamed student posted this letter recently, and between Digg and Slashdot, it gained a considerable following. The letter indicates that the student was disciplined for using Firefox, an act which approaches holy war status in certain circles. The attention prompted the school to issue a response stating that the letter was altered and that the student was not disciplined for the use of Firefox. It does prompt one to question what was altered in the letter posted. I would hope they are talking about something more substantial than the blacked out names. I wouldn’t be too surprised if the Firefox promotional (“…but he told me that it was just a different browser and that he was doing his work. … he insisted that it was a better browser and that he wasn’t doing anything wrong”) was embellished. It seems to me that a teacher would call it insubordination and leave it at that. In any case, the kid caused a lot of attention, hoax or not. It’d be great to hear what the actual alterations were, and how the school handles the situation with the student, after having caused them a massive PR headache.

Warning – long and boring. This is
as much for my reference than anything else.

It started Friday – several weeks
ago. In the evening, one of my drives, a 250GB SATA, threw some
errors. The RAID5 wasn’t terribly concerned, it corrected the reads
and was happy. There were probably less than a dozen errors, and it
didn’t kick the drive from the array. I made a note of it, but
didn’t bother kicking it out manually.

Early Saturday, a 400GB drive dropped
out of the array. This drive does this from time to time, going
utterly unresponsive but fine upon reboot. I re-added it to the
array, and it began to sync up. Chris and I went to the new Circuit
City in Chambersburg, as he needed to get a power supply for
debugging a box lockup issue. I decided to buy one of those
new-fangled DVD burner thingies, as it was probably about time I had
one. Upon getting home, my array was not happy. I had 3 active
members on a 5 device RAID5. Rebuilding the 400GB had sent the
ailing 250GB over the edge, kicking them both out of the array. It’s
a curious thing to see in /proc/mdstat. The metadevice stayed
active, but degraded. Ext3 freaked out and dropped to read-only. I
really would have expected the metadevice to deactivate under those
conditions… or better yet, be very reluctant to kick a drive from
an already degraded array. If only I had kicked the 250GB manually,
this would have been a bit less stressful. So, then the contingency
planning starts. Do I force the array back together, and try
resyncing the 400 again? Will the 250 be so badly corrupted that it
makes more sense to force the mostly-current 400 back in the array
instead of the 250? Should I dd the 250 to another drive, since dd
should at least keep going instead of giving up on the errored
sectors? Not pleasant thoughts or options. SMART data was
indicating that the temperature of the drive was over 60C–hotter
than the box’s CPU. I moved it to another machine for diagnostics,
which didn’t turn up anything. The Hardware_ECC_Recovered was
varying rapidly (not that that necessarily means anything…), so I
decided it was time to be replaced. I ordered a 500G (WD5000YS) and
another Promise SATA-II TX4 PCI card from Newegg. Later that night,
I put the 250G back in the box and tried the resync again. I watched
the resync all night (something like 4am), waiting for it to either
fail, or complete. I wanted to boot the 250G from the array at
completion, so this wouldn’t happen again. Yes, I could have and
should have scripted it. I was worried about my data! The resync
completed successfully with no errors. Seems the 250G was much
happier after it had flagged its bad sectors.

On Sunday, I really couldn’t do
anything about the array, so I started down the second storage path
of death for the week: the DVD-R drive. I installed it in my
desktop, fired up k3b, burned a backup DVD of several years of
photos, and it seemed fine. But I could mount it anywhere. Turns
out that it (k3b and/or growisofs) wants to burn DVD+Rs as unclosed
multi-session discs. Fine. Turned that off, and burned myself
another one. It was fine. It was nice to have something work for
once.

On Monday evening, feeling lucky from
the day before, I tried burning some more photos to DVD, but it was
not to be. IDE errors would start spewing into dmesg, growisofs
(which had elevated itself to a nice of -20) began consuming the
entire machine, making it unusable. I tried different speed
settings, just about any option k3b had to offer. I moved the IDE
cable to a different controller, tried changing cables, anything…
DMA settings, I looked for firmware, but the thing is a no-name OEM
drive probably originally from Lite-On, but their firmware won’t load
on it, and the site supposedly having the firmware genericizer was
down. Of course, I gave up at some point and burned something in
Windows which was fine… ARRGH!

Tuesday was supposed to be the day of
productivity. The new drive and controller arrived, and I installed
them. I spent a little time tooling the partition table and began
the resync. The mirror resync’d very quickly at 30-40MB/s. The
RAID5 resync stayed around 27MB/s when the system was idle, but
dropped considerably otherwise. The old setup would only resync
around 20MB/s, and was otherwise usable. But at 27MB/s, the system
crawled, yet wasn’t using up 100% CPU. I think this is the surreal
PCI bus exhaustion experience… 27*5=135, and 133 is the maximum for
a 33MHZ, 32-bit, PCI bus. But many of my PCI devices (including the
northbridge), are 66MHz capable, and from what I’ve read, 33MHz
devices shouldn’t be holding back the 66MHz ones entirely, but I
couldn’t find out how to test/debug this further. Later I found out
that the 66MHz-capable bit doesn’t mean very much, and what you
really need is a 66MHz-capable PCI bus – which mine isn’t. Myth
wasn’t happy about this, as ivtv wasn’t getting read from fast
enough. The system otherwise felt very sluggish. I left the box up
to resync overnight. I fought with the DVD drive some more, too.

Wednesday morning, I checked on the
status of the resync. But the box had locked up. I rebooted it and
checked the logs. The resync did complete, but sometime later, there
was an unhandled interrupt on the IRQ shared between a SATA
controller and the video card. Linux then disabled the IRQ, causing
all of the drives to fall out incrementally. I brought the box back
up, and had to force the array to be “clean” so that it would
re-assemble (echo clean > /sys/block/md0/md/array_state … there
appears to be no mdadm equivalent of this action. And you have to
write something to the device then for the superblocks to get
updated.) I eventually and experimentally determined that running
SMART commands on the new 500G drive is what causes the unhandled
interrupt. It takes time for the problem to manifest though –
maybe it’s a race condition somewhere. I haven’t found anything in
the kernel mailing lists about this, so I will have to research
further and maybe post about it.

As far as the DVD drive, I tried
different media with equally mixed results. I eventually returned it
to Circuit City and bought a better and cheaper Samsung from Newegg.
It seems to work much better… no spewing of IDE messages. I almost
made myself buy a SATA one, but I didn’t want to buy yet another SATA
controller and risk more problems with compatibility.

And the 250G seems fine now, so it
didn’t get tossed either. Sigh.

I used to like thinkpads. They were black, IBM tried to be nice about Linux support, and the maintenance docs were readily available online. However, my 600e had 2 issues requiring return for repair, 2 more issues requiring parts replacement after warranty, and then had to be retired due to not working well anymore.

I had thought my R40 was doing better, but perhaps not. A month or so ago I had to replace the hard drive (not entirely blameless here… it was transported frequently with a running drive), and now it seems the video subsystem is beginning to collapse. This morning I noticed some “visual artifacts” that didn’t get fixed with a reboot, and aren’t a problem with the LCD, as the VGA port shows them as well. So, guess what that means? A new system board. Fun, fun. I’m seriously considering selling the thing as parts and moving to a Dell D410 or D420, or perhaps skipping a personal laptop and using a work laptop. The next laptop will be small and light, that much I know.