Patshead.com Blog

I just had the displeasure of ordering four SATA hard drives. They're going to find a home in a 1U server that I'm planning to purchase in the next couple of weeks. I ordered the hard drives early because I need one or two large, empty drives so I can make some major partitioning changes to my home file server.

I had some simple requirements for the drives in this new server. They had to be large, reasonably fast, RAID friendly, and they needed to fit in the 1U chassis. It would have been nice to be able to use some server-grade 10K or 15K RPM disks, but they just don't have the capacity I need, and they are a quite a bit more expensive. The best fit for this server are 2 TB or 3 TB 7200 RPM desktop drives.

In the distant past

In the latter days of IDE hard drives, I almost exclusively used drives manufactured by Western Digital. I had reasonable luck with them, and their RMA process was very convenient. They didn't make you jump through hoops to prove that you had a bad drive, and they would cross ship your replacement drive for free.

I once had a drive in my home desktop machine's RAID 5 fail, and I picked up a Maxtor 160 GB IDE drive as an emergency replacement. The drive worked fine for a few months, but then it started dropping out of the RAID due to read and write errors.

Maxtor wouldn't RMA the drive unless I ran their diagnostic software. Since the drive was plugged into a 3ware RAID controller, their software couldn't see the drive. I had to move it to one of the motherboard's IDE ports. Their software miraculously claimed to have fixed the drive. I had to repeat this process three more times over the next several months before it finally admitted the drive was bad. This made me really start to appreciate Western Digital's RMA policy.

Early Western Digital SATA drives were also quite handy, since they had both SATA and IDE power pins. In those days, most power supplies only had one or two SATA port connectors, so this was terribly handy.

I continued to use Western Digital's SATA drives right up until about three or four years ago. I picked up some of their 1 TB, 7200 RPM hard drives to use in a Linux software RAID 10. Those drives liked to randomly drop out of the RAID due to read timeouts, which seems to still be a common problem with many of Western Digital's drives.

Switching to Seagate

I replaced those 1 TB Western Digital drives with equivalent models from Seagate, and I'm still using most of those drives today. I did have to RMA one of them, though. The process isn't quite a slick as Western Digital's used to be, but it wasn't insane like Maxtor's either. Seagate's drives also came with five-year warranties, which beat Western Digital by two years.

Things are looking pretty ridiculous this year

Seagate doesn't seem to have any drives with a five-year warranty anymore. The drives in the capacities and speed that I was shopping for, they were all one-year or two-year warranties. Here's where it seems to get stupid: drives with the same model number can have either a one-year or two-year warranty. They may even have a different number of platters, heads, and entirely different performance characteristics.

In the future

I did buy four Seagate drives today, but I'm not sure I will be buying any more drives from Seagate in the near future. I'm not too enthusiastic about the shorter warranty periods. I have four or five 1 TB Seagate drives in the room with me here, and many of them will still be under warranty after the warranty on these new drives expire.

I'm even less happy about that fact that I have no idea what sort of drives are going to arrive at my door later this week. I don't have any solid data, but the Internet is leading me to believe that I may get some combination of two-platter and three-platter drives. The two-platter drives are slower, and they may not be as well made as the three-platter drives.

It sure feels like a conundrum.

Update: The drives arrived today. If I am decoding the serial numbers correctly, one is the superior 6 head model. The other three are the slower two-platter, four-head model. All four were made in Thailand, and I am pretty sure that is good news.

Update 2013-06-10: Since writing this entry ten days ago, I have almost completely convinced myself that I am not going to use BitTorrent Sync. I just discovered Seafile last night, and it looks like it will fit nearly all of my requirements. I'm currently testing it out on one small, but very active directory. If this test works out, I will try to extend the experiment out to my entire home directory.

Almost two months ago, I looked into the idea of replacing Dropbox with my own self-hosted solution. No single piece of software was able to meet my needs. Since then, I've put some thought into what my goals are. It looks like I will be able to meet the majority of my requirements by combining two or three different pieces of software.

The goals

I want to synchronize the contents of my 25 GB home directory between multiple locations. I would like to be able to replace my existing backup plan with one of these synced copies. That means at least one location has to have file versioning or snapshotting. If I want to be able to eliminate my existing backup plan, at least one of these copies has to be stored securely off-site.

I need a convenient way to share files. I can easily share entire directories using Dropbox, even if the recipient only has a web browser.

I am most definitely addicted to Dropbox's Android app. Automatically uploading photos is extremely handy. I will most definitely need to be able to replicate this functionality.

They all fall short of the goals

I could most likely meet all of these goals using Dropbox, but I don't trust Dropbox with all of my personal data. Dropbox encrypts your data on the server side, and they have all the keys needed to look at your data.

SparkleShare and ownCloud can each meet the first two goals, but not without problems. You can encrypt your SparkleShare repository on the client-side, but then you won't be able to share files. ownCloud has the automatic photo uploading covered on Android, but there are no plans to add client-side encryption to ownCloud.

BitTorrent Sync can easily and securely move my data around, but the data it moves around is stored unencrypted on each end. That means that you have to put a lot of trust in each endpoint.

My actual plans

I should be able to accomplish most of my goals with a combination of BitTorrent Sync and a snapshot-capable file system, like ZFS or btrfs.

Sync and back up my home directory

To start things off, I plan to synchronize my laptop's entire home directory with my home file server. This will require some effort, and it will be a bit of a challenge. My file server does not currently have the capability of taking snapshots, and I don't power it up 24/7 anymore.

I plan to have the server wake up long enough to sync up with my laptop once each day. After the syncing is finished, the server will record a snapshot and shut itself down. The hard drives will only have to spin for a few minutes each day, and it will still be easy to keep my home office cool during the upcoming Texas summer.

Around seven years ago, I replaced a small laptop and a bulky desktop computer with just a single laptop. I haven't had a desktop computer since then, and I don't know if I ever will again. Late last year, my friend Brian very nearly convinced me to build a desktop computer for myself. Having good, solid sync capability for my entire home directory would make it much easier to revert back to using two devices again.

Sharing files

I had originally planned to use ownCloud for this, but I thought of a simpler solution. I am going to create a directory called "Public" on my laptop, and I will use BitTorrent Sync to push it to a web server. That should cover nearly all of my random file sharing needs.

BitTorrent Sync has its own built-in, secure file-sharing system, but the recipient needs to have the BitTorrent Sync client installed in order to receive the files. This option isn't ideal, but I wouldn't be surprised if it ends up coming in handy every now and again.

Camera uploads

This is the other reason I had originally planned on supplementing BitTorrent Sync with ownCloud. The folks over at BitTorrent Labs had an ARM build of their client available since they launched, and there's already an app in the Google Play store to install it for you.

I can probably get away with syncing the photos directly to a web server. The Titanium Backup backups of my phone and tablet are probably using most of my Dropbox space right now. I should be able to eliminate that upload to Dropbox and just sync that backup elsewhere.

Something is missing

That something is the off-site backup. I don't have an ideal solution for this part of the problem yet. I will leave my existing rdiff-backup and Duplicity backups in place until I figure out a better plan.

What is the first step?

I will finally be home, in my own office, sitting in my own chair at my own desk in just under one week. When I get there, one of the first things I'll have to do is find some spare disks to temporarily hold all the data on my file server. Then I can tear down the existing RAID 6 array, and replace it with a ZFS RAID-Z6 volume.

That is actually the hardest part. The rest should be easy, so stay tuned!

I've been trying to remember to implement items from my zsh-dwim notes in my spare time. I managed to implement some apt and dpkg helpers this week.

You must use an HTML 5-compatible browser to view this video.

add-apt-repository -> apt-get update

Every six months I upgrade Ubuntu. I always end up having to add at least one or two PPA repositories, using the add-apt-repository command, every single time I upgrade. Just about every add-apt-repository command is followed by an apt-get upgrade. This transformation was definitely a no-brainer.

which -> dpkg -S -> apt-cache show

This one was a little less obvious, but probably just as useful. Often enough, I'm interested in finding out exactly which package the file returned by the which command belongs to. That usually means either typing dpkg -S and pasting in the path, or recalling the previous command from history and manually adding the dpkg -S and the required backticks.

That just seems like too much work. Now I just hit control-u, and the which command is converted directly into a correctly formatted dpkg -S command.

I ended up taking it one step further. Hitting control-u on a dpkg -S command will transform the command into a slightly buggy apt-cache show $(dpkg -S ) command. I'm not terribly happy with how the end result turned out; it ends up generating a rather verbose command.

You can find zsh-dwim at GitHub.

Buy a good, comfortable, sturdy chair now rather than later

An Aeron would have been a pretty big expense for me fifteen years ago when I bought my first office chair. Just a few years later, though, and the price wouldn't have been too big of a big deal. I should have bought one a decade ago. I'd have been sitting more comfortably the entire time, and in the long run it would have been less expensive than buying a new, cheap chair every few years.

Not everyone finds the Aeron to be a comfortable chair. Fortunately there are plenty of other good chairs to choose from. I'd definitely recommend spending some time sitting in them before you buy one, though.

My Progression of Chairs

At some point when I was in school, probably shortly before starting high school, my parents bought me a wooden office chair like this one. I remember them saying they got it at a yard sale for five or ten bucks. I would guess that I started sitting in that chair sometime around 1990, and I continued to sit in that chair through most of high school. I don't remember it being uncomfortable, but very little is uncomfortable when you're still under 140 pounds.

At some point that chair finally broke, and I upgraded to another wooden office chair, like this one. This one once belonged to my grandfather, and it is still at my parents' house today.

I replaced it with a random, cushioned, high back executive style chair at some point after I got out of high school, most likely sometime around 1997. Over the next decade or so I bet I bought a half dozen of these chairs, each for between $150 and $250. They weren't well made, and I often left them behind or gave them away when I moved.

The story of my Aeron Chairs

I finally convinced myself to buy an Aeron chair in the latter part of 2009. I ordered a used "Cobalt Classic," fully adjustable model through a vendor on eBay for around $450 shipped. The chair was in pretty good shape, and the label said it was manufactured in 1997. The sides where the arms meet the chair were pretty scuffed up, presumably from twelve years of adjusting, and it was a bit squeaky when leaning back.

About a year later, my father had a heart attack. I knew I'd be spending some time up there, so I drove the 900 miles up to my parents' house with the Aeron in the back seat. I was up there for about two months, and in that time we ended up ordering another Aeron for my father, and I left my "Cobalt Blue" Aeron behind for my mother.

Shortly before I left to return home I found another chair on eBay for myself. This one had the more common "Carbon Classic" color scheme, which is a black chair with a dark gray weave in the pellicle. This one was "new"; I believe it was some sort of floor model or something, fully adjustable, and has the newer levered locking mechanisms on the arms.

I have been visiting my parents again for the last six weeks or so, this time for back surgery. I am in my old bedroom, at my 20-year-old desk, sitting in that same "Cobalt Blue" Aeron chair. I'm not exactly sure why, but it no longer has a squeak when I lean back.

I still miss the blue chair a bit. It was a very nice-looking chair; the "Carbon Classic" is pretty boring in comparison. I'm pretty certain it is the same color as Sheldon Cooper's Aeron chair.

Why the Aeron chair? Aren't there better chairs now?

I'm certain there are better chairs available now. I can't say I've ever sat in any of them, though. The major selling point of the Aeron was the fact that I'd spent some time sitting in one in the past and I already knew I liked it. After that, the price and build quality of a used Aeron made a lot of sense.

My "Carbon Classic" Aeron chair has been in my possession now for almost four years. It looks and feels exactly the same as it did when I first got it. I expect to be able to say the same thing ten years from now.

Some people dislike the Aeron's pellicle seat. More specifically, they don't like the hard edge at the end of the seat. Some people say it cuts off the circulation in their legs. I'm a pretty heavy guy, and this has never been a problem for me. In fact, I don't think I'll ever buy an office chair with a cushion again.

The breathable pellicle seat is more comfortable on warmer days, and it even helps prevent swamp ass:

This little Perl program has been sitting in my ~/bin directory for almost a year now. It has quite a few features that I don't use very often, and some of those rarely used features seem to have a lot of bugs. Even so, I figured that it was time to throw it up on GitHub.

Where did shstack come from?

Fairly often, I get into a situation where I am performing different parts of the same task in two or more terminal windows. A lot of times I wish I had easier access to path and file names in the other window. I thought I'd write up a little script to automate the process.

I very quickly realized that it would be handy to be able to work with more than one file at a time, so emulating pushd and popd made a lot of sense. Since I was already planning to store the stacks in a file, I decided that they might as well be persistent.

What can you do with shtack?

I store the DNS names of some important servers in a stack called servers. I can use shstack's built-in for command to ping them all to see if they are alive.

Why reinvent the for loop?

I have two reasons. The first is shstack's "safe" for loop (sfor). The sfor loop works like a regular for loop, but it removes each item from the stack when the command is completed successfully. You might use this if you wanted to rsync a directory to multiple servers. That way, if there are any errors, you can easily fix the problem and run the command against the problematic servers again. I didn't end up using this as often as I expected, though.

I also wanted to be able to write very simple commands that used items from multiple stacks. In this example, I'm using rsync to back up three directories to two different remote servers:

Real world example: one of my backup scripts

I use Duplicity to make a backup of important parts of my home directory once a day. I have a script that runs right after this to copy those new backups out to a couple of servers out on the Internet. The script pulls those server names from a stack called rsyncDestinations:

#! /bin/bash

SRC=/mnt/platter/Backups/duplicity
source ~/.keychain/zaphod-sh

s for "echo Syncing $SRC to %rsyncDestinations%;
       nice rsync -va --partial --delete --progress $SRC %rsyncDestinations%
      "

The future of shstack

Shstack definitely has some bugs. Some of them are just cosmetic, while others are pretty serious. I was going to record a screen cast showing off how sfor works, but I immediately noticed that sfor doesn't correctly handle file names with spaces. I was pretty surprised by this because I'm usually pretty good about making sure my code is space friendly.

Obviously, I'd like to fix some of these bugs. I'm sure I'll find others along the way too. I'd also like to write some documentation. I don't think the help screen is verbose enough to actually explain how to work it.