The automated backup strategy that I’ve been using for the past three or four years was very solid, and it served me very well. I was using rdiff-backup to perform daily backups of my entire home directory to local storage, and I was using Duplicity to create and upload encrypted backups of my most important data to a remote server every day.

This was very stable and very bandwidth-efficient, but each backup only ran once a day, and rdiff-backup and Duplicity are both very heavy on I/O and CPU.

Moving my backups to the cloud

I started exploring various self-hosted cloud storage solutions a few months ago. After much consideration, I decided to use Seafile as my cloud storage solution. I had to rearrange my home directory a little to get everything important synced up to my Seafile server, but once I did, I realized that I didn’t need my old backup plan anymore!

A good backup plan needs to include history. It might take days or even weeks to notice a corrupt or missing file. It is important to be able go back into your backup history and recover that file Seafile definitely has me covered in this case. Seafile lets me set the number of days of history I want to hang on to for each individual library. My rdiff-backup and Duplicity backups only recorded a snapshot of my files once each day, while Seafile does that constantly throughout the day as files change. I consider that to be a very big win.

A good backup plan also needs to move your files off-site Seafile has me covered here. Changes to my files are pushed up to my server almost immediately after I save them. My computer could get fried right now, and I would probably only lose this paragraph.

Seafile doesn’t just give me a backup of my data, though. It is also keeping my important data synchronized between my laptop and my desktop. That not only gives me an extra full copy of my current data, but it also means that I don’t have to remember to take files with me when I leave the house.

Don’t forget to back up your self-hosted cloud!

In my opinion, a proper cloud-based storage solution has redundancy. Of the self-hosted cloud storage solutions that I checked out, the only ones that offered any real redundancy didn’t have client-side encryption. That made them pretty useless for me.

I don’t care if my Seafile server becomes unavailable for a while, but it is very important to me that I don’t lose all history that it is storing for me. I have some existing infrastructure in place to help keep my Seafile server safe.

• My Seafile virtual server is stored on a RAID 10 to help protect against hardware failures
• All my virtual servers are backed up to another local server every night
• Those nightly backups are then synced off-site

This means that two copies of my current data exist at almost any given time: one on my local machine, and one on the Seafile server. I get even more redundancy if my laptop is powered up. It also means that my backup history is stored in three places: on the Seafile server, on the local backup server, and on the off-site backup server.

I’m probably doing a better job of managing my backups than most corporations at this point.

How safe is Seafile?

I’ve only been running Seafile for a little more than a month so far. I haven’t had any data loss, and I’ve been able to pull files out of my history. One month isn’t a long time. I’ll definitely report back regularly to let everyone know how things are going.

So far, though, things are looking pretty good!

My friend Brian decided that he wants to build his own arcade cabinet. He’s been poking around on Craigslist for a couple of weeks trying to find a decent shell to rebuild. He finally found one that he liked, so we pulled the seats out of my minivan and drove to Arlington to pick it up.

I think he did alright. The cabinet was already in such a state that we had absolutely no idea what game it used to be. In fact, we thought that it might have started life as a scratch built MAME cabinet.

We were wrong about that. Going by the shape and dimensions of the cabinet, and the original holes on the underside of the control panel, we are pretty certain that it is a Main Event cabinet. It is already so far gone from its original state that we don’t have any reservations about cutting into it and converting it to a MAME cabinet.

I’m pretty excited. It has been quite a few years since I finished up my own arcade cabinet build, and I haven’t had the opportunity to work on anything like this since. I think it is going to be a lot of fun helping to bring this cabinet to life!

The desk – Bush Series A – Left L-Bow

I’ve been using this desk for a long time, probably since around 2006. It isn’t made of proper wood, but it is quite heavy-duty. The three legs are 1” sheets of laminated particle board, and the surface is a heavy, 1”-thick slab of laminated MDF. My desk is quite large, and it doesn’t fit through doorways when fully assembled.

The desk is roughly 60” wide by 44” deep. It is big enough that I can just rotate my chair to the right if I need to work with my laptop, and back to the left to use my desktop. I often tend to primarily focus on one monitor at a time, but the primary display changes depending on what I am doing. I’m pretty fluid in this regard, but I do almost all of my text editing and gaming on the right-hand monitor.

When I switch my focus from one monitor to the other, I rotate my chair and move the keyboard over so it is centered on that display. It is more comfortable to keep your neck straight most of the time.

I upgraded the desk’s grommet by adding a powered USB hub and some power outlets. I use the USB hub to charge my phone, and it comes in handy for flash drives and USB hard drives. The power outlets are plugged into my UPS, and they come in handy all the time.

The surface of the desk is very smooth, so I don’t need to use a mouse pad. It has taken a bit of a beating over the last half-dozen years, so there are some scratches and dings.

It is shaped to fit in a corner, but I have it in the wrong corner. It made sense to rearrange things in here a bit after I connected a wall-mounted LCD TV to my arcade cabinet. I can now easily see both of my monitors and the TV from where I’m sitting.

The dual monitor stand

I built this monitor stand three years ago. The use of PVC combined with steel pipe is pretty unique, and not without its problems. The initial design ended up being a failure, and I ended up wandering around Lowes trying to come up with a way to salvage the design.

It turned out that the screw holes in ¾” PVC j-hooks match up perfectly with VESA mounting points. The rest of the design fell into place from there. I didn’t end up having to glue any of the PVC; friction was enough to hold it in place. I only had to tweak it once every few months, but that wasn’t really much of a problem.

That is, until my new friend Harley showed up. He likes to attack mouse pointers, rub up against the monitors, and shimmy his way underneath them. He doesn’t mess them up as often as he used to, but I still have to straighten things out every few weeks.

I’m very happy with the way the steel pipe flange bolts so solidly to the desk. I’d like to keep that and upgrade the rest of the mount using unistrut, but I’m going to put that off until my next monitor upgrade.

The mechanical keyboard

The keyboard is an IBM Model M PS/2 keyboard. This particular one happens to have been born in 1993. I have some plans for this keyboard. I’d like to at least replace the old, coiled cable with a nice fabric-jacketed cable. I also miss the volume control knob on my old Bluetooth keyboard. I might try to add a nice analog volume knob using an Arduino.

For now, though, I’m just happy to be using a mechanical keyboard again.

The computer

I just recently built a new Linux workstation to replace my aging laptop. It is an AMD FX-8350 with 32 GB of RAM, 128 GB Crucial M4 SSD, and an older, power-hungry NVidia GTX 460 video card. I am also mirroring a pair of 7200 RPM 1 TB that I had lying around; they are an excellent replacement for the second 500 GB drive that was in my laptop.

The video card might seem a little outdated and out of place in this machine, but it is more than fast enough to meet my needs. It is at least 10 times faster than the NVidia GT 230M that was in my laptop, and it runs nearly every game I play at well over 100 frames per second with all the settings maxed out. I won’t bother upgrading it until I want to play a game that won’t run smoothly.

The headset

I’m using a Monoprice gaming headset. They’re one of the few gaming headsets that not only have a removable microphone, but the microphone isn’t obnoxiously huge. They also happen to be just big enough for my over sized cranium, which is a nice bonus. They’re also solid black and very plain looking, which is exactly what I was hoping to find.

They’re the only set of huge, closed, monitor-style headphones that I’ve ever owned, so I’m not sure how useful my opinion is. They’re comfortable enough that I don’t mind wearing them for an hour or more at a time. They sound pretty good to me, and they’re louder than I’d ever need. They also happen to be reasonably priced, which certainly doesn’t hurt.

The Aeron chair

I love my Aeron chair. I’ve only had it for about four years so far, and I really wish I’d bought one sooner. Everyone should invest in a quality office chair. I should have bought an Aeron at least ten years ago. I would still have it today, and it would still be under warranty.

The three Aeron chairs that I’ve bought are all used or refurbished. One of them was manufactured in 1997. That one has a very slight wobble to it, but is otherwise in very good shape. Computer hardware becomes outdated pretty quickly. Quality office furniture lasts much longer, and is worth investing in sooner rather than later.

The UPS – an old APC 650

I’ve had this UPS for over a decade. I bought it used off of eBay shortly after I moved to Irving, TX in 2001. We used to get a lot of brownouts that would cause my computer to reboot, and a UPS was a good fix for that. My APC 650 is on its third or fourth replacement battery, and I won’t be surprised if it is still running in ten more years.

The cocktail arcade cabinet and television

The arcade cabinet is one of my favorite pieces of hardware in my office. My brother-in-law John and I probably spent nearly 100 hours building it, and we’re both very proud of how it turned out.

It is not just a fun machine for playing classic video games. It is also has room for seven hard drives, so I used it as my home file server. I also started using it as a media player now that it has a wall-mounted TV connected to it.

I’ve already written many thousands of words about the build process. Here are some useful entry points into that series of articles:

• Cocktail Arcade Cabinet Upgrade: Part 3 – The Conclusion
• Cocktail Arcade Cabinet Build: Part 1 – Planning and Testing
• Cocktail Arcade Cabinet Build: Part 10 – The Finished Cabinet

Whiteboards

I love whiteboards, and I am a big fan of using melamine sheets as inexpensive whiteboard material. My previous home office had 84 square feet of whiteboard. The layout of my current home office just didn’t have room for all three of my old whiteboards because the arcade cabinet and desk both block quite a bit of wall space.

I now have one 4’ by 8’ and one 4’ by 5’ whiteboard, and that is still quite a lot of square footage. The nice thing about having a lot of whiteboard space is that you don’t have to erase things as often. I bet we spent the better part of a year drawing out all sorts of arcade cabinet designs before it evolved into what we actually built. There were often at least two generations of design on the walls at any given time, and it was nice to have enough space to draw a life size diagram of the tabletop and controls.

I haven’t had a chance to update zsh-dwim in quite a while. I had a tiny spark of inspiration for a new transformation tonight, so I went ahead and implemented it. I also picked two other easy ideas off of my to-do list to implement, since the new idea was so tiny.

Tonight’s new idea

I was manually installing a couple of packages using dpkg. There were a couple of missing dependencies, so I immediately knew that I’d have to run apt-get -f install to pull those dependencies down. I was surprised that I was smart enough to realize that this was exactly the sort of case I was thinking of when I created zsh-dwim.

When dpkg --install fails due to missing dependencies it exits with a status of 2. Now, if you hit control-u when that happens zsh-dwim will be smart enough to give you back the apt-get -f install that you are looking for.

watch –> watch -n 10 –> watch -n 30 –> watch

I don’t know about you, but adjusting the interval of the watch command is something I do quite often, and I rarely remember to make the change until I’ve already punched in the entire command. With this new transformation I can just hit control-u repeatedly to cycle through various intervals.

modprobe –> modprobe -r –> modprobe

I’m not entirely certain how useful this one will be, but it was on my list. It was also very easy to implement, and I find these circular transformation interesting. We’ll see how it works out, if I manage to remember it is even there.

The motivation

I’ve been thinking about upgrading all year. I almost did back in January, and I wish I had decided to do it then. Memory prices have just about doubled since then. I’ve been using a laptop as my primary workstation since 2006, and I wasn’t sure if I wanted to switch back to a desktop. Since I’ve had some recent success using Seafile to synchronize a huge chunk of my home directory, I figured I could get away with using two different computers again.

I’ve had my current laptop for over three years now, so I figured it was time for an upgrade. It is fast enough for almost everything I do, but I’ve been playing more games lately and the video card just isn’t fast enough. I have the graphic settings turned way down in most games.

Some of the games I play, like Team Fortress 2, run at perfectly acceptable frame rates. I’m not so lucky in other games. Upgrading to a desktop gives me a guaranteed video card upgrade for free.

The goal

History and Moore’s law made me think that I’d be able to build a reasonably priced Linux desktop that was at least three or four times faster than my laptop. I was very mistaken. Moore’s law doesn’t seem to be translating directly into performance like it used to.

Doubling my memory and processor speed of my Core i7-720QM laptop turned out to be pretty easy to achieve, and I didn’t have to break the bank to do it.

Parts List

• AMD FX-8350 8-core CPU – $195
• MSI 970A-G43 Motherboard – $70
• Corsair XMS3 (32 GB, 4 sticks) – $232
• Topower ZU-650W power supply – $73
• NZXT Source 210 case – $40
• Crucial M4 128 GB SSD – $105
• NVidia GTX 460 – free

Optional:

• Arctic 7 CPU Cooler – $24 (much quieter than the stock fan)

Total cost: $715

A slightly faster equivalent to the video card I used would be an Nvidia 650. That would bring the total cost up to about $825.

All the devices on this motherboard are working just fine for me on Ubuntu 13.10. I don’t have any USB 3 devices, though, so I am unable to confirm whether they are working correctly or not. I can confirm that the USB 3 ports on the rear panel are working just fine with USB 2 devices.

The processor and motherboard

I’ve had my sights on the AMD FX-8350 for quite a while now. To get a processor from Intel with comparable performance you’ll end up paying around $100 more, and you’ll also have to pay more for the motherboard. I could have spent $150 to $200 more on Intel parts for about a 15% performance boost, but that didn’t seem like a good value.

Should the FX-8350 really be referred to as an eight core processor? Probably not. It sounds to me like it has eight nearly complete integer units and four completely floating point units. I’d like to do a bit of testing to see exactly how close to complete those eight integer units actually are, but for now, I am going to say that the FX-8350 is more like an eigth-core processor than a four-core processor with hyper-threading—at least as far as integer operations are concerned.

When my friend Brian built his FX-8350 machine, he ended up using a motherboard with the 990 chipset. At that time, the motherboards with the 970 chipset weren’t shipping with BIOS support for the FX-8350. This isn’t a problem anymore, so I was able to choose a less expensive motherboard.

I opted for the MSI 970A-G43. I chose this motherboard because it was one of the least expensive 970-based boards from a manufacturer I trusted. I’m much more impressed than I thought I was going to be. I knew before placing my order that it had six SATA 3 ports and room for 32 GB of RAM. When I opened the box, I was surprised to see solid capacitors on the board. I’ve never actually had a capacitor failure, but it was still nice to see.

32 GB of memory

I definitely went overboard on memory. My laptop has 16 GB of memory, and almost 10 GB of that is usually being used as disk cache. I could function relatively comfortably most days with only 8 GB. It doesn’t matter, though. Memory is still cheap enough that it made sense to me to max it out, even though it is almost twice as expensive as it was late last year.

The NZXT case

This NZXT Source 210 is the second case from NZXT that I have seen. They are both quiet and well made. They lack some features, though. I usually prefer cases with easier to access, 90-degree rotated 3.5” drive bays, but I’m willing to live without them at this price point.

The Topower ZU-650W power supply

The power supply was another pleasant surprise. The spare video card I have has a pair of PCIe power connectors that need to be populated, so I wanted to find a power supply that could meet that requirement. The Topower ZU-650W just happened to be on sale while I was placing my order, and I am lucky that it was.

All of its cables are wrapped in sleeves, so it is easier to manage that potential rat’s nest of wires. I was also surprised to see that it came with five Velcro cable ties. The Topower has one feature that really surprised me: a “turbo fan switch.” I haven’t had any sort of “turbo” button on any of my computers in 20 years!

The solid-state drive

I didn’t have to buy the 128 GB Crucial M4 SSD. I simply moved it from my laptop into the new machine, and it booted right up. I included the price in the parts list to help paint a more complete picture of the build.

The video card

Late last year, a friend of mine built himself a new computer very similar to this one. He donated his old video card to me for use in my arcade cabinet. I couldn’t use it in the arcade cabinet because it requires two PCIe power connectors, and the power supply in the arcade table only has one.

This card is an NVidia GTX 460. With its 336 CUDA cores, this card should be around ten times faster than the mobile NVidia card in my laptop. This should be fast enough for the foreseeable future. It is doing a fine job running all the games I have. That isn’t too surprising, since the games I play are all pretty old.

I’m getting 100 to 150 frames per second in Team Fortress 2 with some antialiasing enabled and all the rest of the settings maxed out. Killing Floor still drops down below 60 frames per second when things get busy; I’m pretty sure the Linux port is just buggy. Maps like “Abusement Park” that were nearly unplayable on my laptop are running just fine now, though. I think this video card will keep me happy for quite a while.

If I did have to buy a video card today, I would choose the NVidia 650 Ti. It is at a very nice point on the price/performance curve, and I’ve seen the Nvidia 650 Ti Team Fortress 2 at 2560x1440 with all the settings maxed out. That is more than fast enough for my own purposes. You could save a little money with the NVidia 650, but it has half as many cores as the 650 Ti, so the bang for the buck isn’t as good.

Benchmarks

I was very interested in seeing just how far I’ve come from my laptop. I tried to come up with a few benchmarks that would help gauge just how much of a real world performance increase I would see.

Geekbench – Laptop: 5,950 FX-8350: 12,981

Geekbench is a pretty good benchmark of CPU performance, and I relied on Geekbench’s results browser very heavily while I was shopping. I was really hoping to triple my laptop’s processor performance, but I quickly learned that the required hardware was pretty expensive.

My first-generation, quad-core i7 laptop manages a Geekbench score of 5953. I didn’t want to bother upgrading unless I could double that score. I was a little worried, though, because the scores for the FX-8350 cover a range from 9500 to 13,500. I was hoping to reach 13,000.

This wide range of scores for the FX-8350 was my primary motivation for this write-up. I have no idea what is wrong with those poor FX-8350 machines that are scoring under 10,000, and I was a bit worried that I would be down there with them. I’m happy to be able to report that the FX-8350, paired with the very reasonably priced MSI 970A-G43, performs very well.

I was happy to see that the parts I chose for my new Linux desktop were able to pull off a Geekbench score of 12,981. That’s close enough to 13,000 for me, and it is better than the majority of scores for FX-8350 machines. This is definitely good enough for now, but a tried out a small bump in CPU multiplier, and that brings that score up to 13,649

Linux kernel compile time – Laptop: 3:48 FX-8350: 1:40

I did run this test on both machines in RAM on a tmpfs file system. This seemed more fair. My drives are encrypted, and I didn’t want the laptop’s lack of encryption acceleration to be a factor.

I ran make defconfig && /usr/bin/time make -j 12 on fresh copy of version 3.10 of the Linux kernel. I did some testing way back when I bought this laptop, and determined that 12 jobs was pretty close to ideal. I did make a run with -j 16 on the FX-8350, but I saw no improvement.

The laptop completed the task in 3:48, while the FX-8350 took only 1:40. That’s 2.28 times faster than the laptop and is in line with the Geekbench results.

Note: You don’t need more jobs than you have cores when compiling from a RAM disk, since the compiler never has to spend any time waiting on the disk. A make -j 8 gives virtually identical results in this test.

1080p h.264 encoding with Handbrake – Laptop: 9.03 FPS, FX-8350: 26.71 FPS

I haven’t actually transcoded much video in the last two years, but I’ve had to wait on this kind of job often enough that this seemed like a useful test. To save some time, I encoded chapter 15 of the Blu-ray “Up” using my slightly modified “high profile” settings in Handbrake. Chapter 15 is roughly four and a half minutes long, so I didn’t have to spend too much time waiting for results.

My new FX-8350 workstation is almost three times faster than my laptop in this case. The laptop only managed 9 frames per second, while the FX-8350 pulls off 26.7 frames per second. That’s fast enough to transcode a 24-frame-per-second Blu-ray movie in real time, even using these “high profile” settings.

openssl speed aes – Laptop: 60 MB/s, FX-8350: 210 MB/s

This test is a bit flawed. I was hoping to see how much advantage the AES acceleration instructions would give the FX-8350, but the openssl package that ships with Ubuntu doesn’t support them. The FX-8350 still manages to pull numbers that are over three times faster than my old laptop.

Laptop:
 type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
 aes-128 cbc      76225.62k    82501.14k    85311.70k    85539.62k    84862.54k
 aes-192 cbc      65285.86k    69284.52k    71094.54k    70744.36k    71078.61k
 aes-256 cbc      56213.70k    59583.91k    61326.30k    60588.61k    60864.64k

FX-8350:
 type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
 aes-128 cbc     112766.76k   119554.97k   123908.45k   280502.03k   285362.86k
 aes-192 cbc      95632.70k   100571.26k   103522.55k   238100.82k   242368.13k
 aes-256 cbc      82652.83k    86610.07k    88366.59k   207808.64k   210090.55k

SSD and whole disk encryption performance

These are the bonnie++ results, including my previous benchmarks from an older article:

Version 1.03c                  ------Sequential Output------- --Sequential Input--  --Random-
                               -Per Chr- --Block-- -Rewrite-- -Per Chr- --Block---  --Seeks--
Machine                   Size K/sec %CP K/sec %CP  K/sec %CP K/sec %CP K/sec  %CP  /sec %CP
Laptop, M4, no encryption  16G   954  98 183205 20 105877   9  4858  99 327088  16  4306 120
Laptop, M4, aes            16G   604  94 152764 16  63475   6  4064  98 145506   6  2380  46
FX-8350, M4, aes            6G   642  99 178581 28  82598  25  3124  99 300500  15  4234 104

The sequential block output has mostly caught back up with the Crucial M4’s unencrypted speed. Sequential input very nearly caught up. I am a little bit disappointed in that, though. I was expecting the SATA 3 ports on my new MSI 970A-G43 motherboard to allow the read speeds to surpass the SATA 2 limited 320 MB/s. I must still be hitting a decryption bottleneck, even with the AES-NI kernel module loaded.

I’m still very pleased with my Crucial M4. Its price and performance are both good, and the drive is still performing well after nine months of hard use and random benchmarks.

The verdict

The new hardware easily exceeded my performance goals. The grunt-work tasks that I usually have to wait for are running two or three times faster than before, which should save me quite a bit of time.

All the games I play are running faster, and they look better to boot. The video card I have is fast enough for now, and it is nice to know that I’m just a video card upgrade away from having a pretty powerful gaming machine.

I have to say that I am very pleased with this build. I now have more performance than I actually need, and I feel that I got plenty of bang for the buck.

Update 2013-07-16: I have two new pieces of information. I ended up with one bad stick of RAM. I have it boxed up and ready to ship back. I’m not in a hurry to find a UPS drop box, though, because 24 GB is still more than I need.

I also swapped out the stock AMD heat sink and fan combo. It is quite loud. I ended up replacing it with an Arctic 7 CPU Cooler. It was very reasonably priced, and it doesn’t get nearly as loud as the stock cooler does when it spins up to full throttle. The computer isn’t as quiet as my laptop, but that isn’t surprising. The PSU fans are now the loudest thing in there, and one of my spare hard drives I stuck in there seems surprisingly noisy.

Update: Everything arrived. I was a little worried about the push buttons. The USPS had them marked as delivered on Friday, but they weren’t in the mailbox until Monday. Other than that, everything arrived without incident. I just wish I ordered the resistors earlier. They arrived with the last group of items, and everything else was useless without them!

I finally took the plunge and bought an Arduino development board. I ended up ordering a knock off Arduino Uno board through eBay for $12.99 shipped. The board arrived a few days ago, and I don’t have anything useful to hook up to it yet.

I ordered an assortment of various electronics components, most of which are shipping out of Asia. This was significantly less expensive than buying locally or buying something like the Arduino Starter Kit. I just have to be patient, and I have to hope everything successfully makes the journey across the pacific.

My list of components so far

From various eBay sellers:

• Arduino Uno clone – $12.99
• Breadboard + 65 Jumpers – $6.18
• 100 micro push button switches – $4.43
• 10 10k OHM NTC Thermistors – $2.50

From dx.com:

• 10 5mm Red LEDs – $1.16
• 10 5mm Green LEDs – $1.35
• 10 5mm Blue LEDs – $1.25
• 10 3mm IR LEDs – $1.40
• Variety pack of 400 1/4W resistors – $5.00
• Variety pack of 120 electrolytic capacitors – $4.80
• Variety pack of 210 ceramic capacitors – $5.70

Total cost: $46.76

I didn’t buy everything that is in the Arduino Starter Kit, but I think I made some pretty good selections. I don’t need any of the motors or the LCD display for any projects that I have in mind, and I have some random DC motors, potentiometers, and laser pointer parts in my junk box.

Some project ideas

When you have a hammer, everything looks like a nail. Every problem I see lately looks like something that can be fixed with an Arduino. I don’t expect to build all of these things, but here is a list of some of the things I’ve come up with so far:

• IR blaster, so the arcade cabinet can control the TV
• Web-controlled IR blaster for the living room—a Logitech Harmony on steroids?
• Automatic, light sensing, motorized window blinds
• Kill-a-watt data logging
• Bias lighting behind my monitors

I also have a few potential uses for an Arduino in my keyboard:

• RGB LEDs for notifications
• an analog volume control knob
• a low-level “panic” button (sometimes games turn off my external monitors on me)

I may have chosen the wrong Arduino

I’m pretty sure that it would have been better to go with an Arduino Nano. The Nano plugs directly into a breadboard, and it looks like it would be quite a bit more convenient for prototyping. I will stick with what I have for now, but I won’t be surprised if I add an Arduino Nano to my toolbox in the near future.

My first attempt at programming the Arduino

An Arduino is rather boring all by itself. It was sitting on my desk for quite a few hours before I learned that the board has an integrated LED connected to pin 13. Shortly after I figured that out, I had it programmed to blink out an S.O.S. on that LED:

A few weeks ago I decided on precisely how I was going to implement my own self-hosted cloud storage, and I even wrote up a blog entry describing what I was planning to do. I discovered Seafile the very next day, and I knew that I was going to have to throw my entire plan out the window.

Out of all the options I’ve investigated so far, Seafile is the most Dropbox can also be configured to keep a history of changes to your files. That means it can be used as a replacement for a significant portion of my backup plan.

Scaling back

I did make an attempt at syncing my entire home directory to Seafile. It was an epic failure. Seafile seemed to grow exponentially slower as it indexed more and more of my data. It took about six hours to index the first 19 GB. Eight hours later, and it was barely past the 20 GB mark.

Seafile slows down as the number of files in the library grows. My home directory has over 275,000 files, which seems to be about 200,000 files too many. Most of the solutions I investigated seem to have similar problems, so syncing my entire home directory is probably not going to be a viable option.

Seafile stores your data in “libraries”, and it lets you sync existing directories up to these libraries. I ended up creating a library for each of the directories that I currently back up on a daily basis, and I also created a more generic library that I called “Seafile”. I’m using the “Seafile” library much like I’ve been using my “Dropbox” directory.

These have been syncing for over a week now, and except for a few problems, I am very happy with the results. None of these libraries have more than about 25,000 files.

Some things to watch out for with Seafile

I ran into two problems, but they are both easy to rectify.

Don’t auto-sync directories that change constantly

I am currently syncing approximately 6.5 GB of data. Four or five days after moving this data into my Seafile libraries, I nearly ran out of space on my Seafile virtual server. Seafile was eating up over 60 GB of space, and it had used a comparable amount of bandwidth.

I was syncing my Pidgin log files and configuration, and also my ~/.config directory. Most of the bandwidth and history space was being used by my constantly updating Pidgin logs and Google Chrome’s cookie database. When I disabled auto-sync on these libraries, the Seafile server dropped from a constant 200 to 300 KB per second of network usage to zero.

This was not a good permanent option, though. I definitely wanted to back up these two directories. Seafile’s command line tools don’t seem to have a way to force a sync on these libraries. I was able to cheat my way around that. I stole the URL from the “sync now” link in the local web interface, and I am querying it every 6 hours from a cron job.

1
2
3
4
5
6
7

#! /bin/bash

## dot-config
curl 'http://127.0.0.1:13420/repos/operation/?repo=61811848-0afd-45a2-982e-009193ac1a8e&op=sync'

## dot-purple
curl 'http://127.0.0.1:13420/repos/operation/?repo=09e7bda0-950f-4bc2-acb1-f8cfc5752010&op=sync'

Server side garbage collection

You have to periodically stop the Seafile server and manually run the garbage collector. That was the only way I could get rid of the 60 GB of cruft my server had collected. If I were still accumulating 10 GB of useless data every day, this might be problematic.

It now seems to be growing at a much more acceptable limit, though, so I’m not too worried about it. I’ll keep an eye on things for a month or so; then I’ll have a much better idea of how much bandwidth and disk space Seafile is going to eat up on me.

Performance

I didn’t think to run a stopwatch until I was syncing my final directory. It was 5 GB of photos. I kept an eye on the throughput numbers in Seafile’s local web interface. It would sit up at 4.5 MB per second for a while, and then it would idle for a while. Sometimes it would sit in between somewhere for a while.

I do know that the entire process from indexing to uploading all my photos came out to an average speed of around 2.5 MB second. I’ve watched Dropbox.

My Seafile control script

I put together a little script to make starting, stopping, and garbage collecting a little easier and safer. I was worried that I would have to run the garbage collector repeatedly, and I wanted to make sure I wouldn’t accidentally do that with the service running.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

#! /bin/bash

SEAFILE=/home/user/haiwen/seafile-server-1.7.0

ulimit -n 30000

case "$@" in
  stop)
    cd $SEAFILE
    ./seafile.sh stop
    ./seahub.sh stop
  ;;
  start)
    cd $SEAFILE
    ./seafile.sh start
    ./seahub.sh start-fastcgi
  ;;
  gc)
    cd $SEAFILE
    ./seafile.sh stop
    ./seahub.sh   stop
    cd seafile
    export LD_LIBRARY_PATH=./lib:${LD_LIBRARY_PATH}
    ./bin/seafserv-gc -c ../../ccnet -d ../../seafile-data
  ;;
esac

My installation is running under nginx for SSL support. If you’re not running under a standalone web server you will need to change ./seahub.sh start-fastcgi to ./seahub.sh start.

Late last year, I went shopping with my friend Brian to help him pick out a mechanical keyboard. We tried our local Fry’s, but they didn’t have a very good selection, so we hopped on 75 and drove down to Microcenter. They had a much better selection. We got to try out keyboards with almost every variety of Cherry switch, except the Cherry MX Clear switches, and even one keyboard with ALPS switches.

Brian didn’t end up taking home a keyboard that day, but he did decide which Cherry switches he preferred. He did some more research, thought about it for a few days, and he ended up ordering a nice keyboard with Cherry MX Black switches.

I was hoping to like Cherry’s blue switches

Shopping for mechanical keyboards, and trying out Brian’s new keyboard had me really missing my old IBM Model M keyboard. The blue switches from Cherry are supposed to feel pretty similar to the buckling spring switches in the Model M, but they are supposed to be a little less stiff and a lot quieter.

They are certainly quieter, but I thought they felt absolutely terrible. The buckling springs just feels so much smoother. The transition at the mid-point of the key press on the Cherry MX Blue switches feels really awful. They just feel way too crunchy.

The Model M

I used my Model M for nearly a decade, right up until around 2006 when I retired my last desktop computer. I certainly missed it, but it was much simpler using a laptop at two different desks without having to dock it at two sets of keyboards and monitors.

Things changed a bit when I bought my current laptop. This laptop spends most of its time connected to a keyboard, mouse, and two monitors. I actually use my old Model M keyboard with this laptop for the first week. The cheap PS/2 to USB dongle that I had wasn’t up to the task, though. I’d see the keyboard LEDs flash every so often when the dongle reset itself, and some important Emacs key combinations just weren’t possible.

The two choices I had at the time were to either order a better USB dongle or unpack the Dell Bluetooth keyboard that I had in closet. I opted to give the Bluetooth keyboard a shot, and I used it for three years.

It was a terrible keyboard. Just like all rubber dome keyboards, it felt pretty darn sloppy. Its space bar was also extremely loud; even louder than the Model M. I can’t believe I used it for three years. On the up side, at least it looked nice not having any wires on my desk.

My “new” Model M keyboard

I’m pretty sure the keyboard Chris is using right now is my old Model M. I was tempted to take it back for continuity’s sake, but I’d hate to steal her keyboard. Instead, I am using my spare Model M, and it isn’t perfect. The cord has a tear in the jacket, it has two wrong key caps, and one key cap is completely missing.

I saw three Model M keyboards when I was visiting my parents, so replacement key caps aren’t too far away. I’m thinking that I’d like to lop off the cord, stick the USB dongle inside the keyboard, and convert this Model M to USB. Then I could buy or make a nice cloth jacketed cable for it.

Speaking of USB dongles, I ended up buying the big, beefy dongle recommended by ClickyKeyboards.com. It is quite gigantor, but it is doing a splendid job. The keyboard hasn’t reset once, and every useful key combination that I can think to try has worked. I couldn’t ask for anything more.

Wireless Model M?

I keep talking about building a wireless Model M keyboard, but I just haven’t had enough gumption to take on the project. It turns out that the controller from a particular Logitech wireless keyboard for the Wii has a matrix that almost perfectly matches the matrix in the Model M! The only mismatch is the slash and asterisk on the number pad; they are transposed. I could certainly live with that.

I saw this hack in a forum post somewhere, but I can’t seem to find it anywhere. It even had diagrams showing the layout of the matrices of both keyboards.

The thing that has been holding me back is the lack of availability of the correct Logitech keyboard. It hasn’t been manufactured in years, and it is pretty hard to find. Especially at a reasonable price.

At the moment, I am just thrilled to be typing this on a proper keyboard.

I just had the displeasure of ordering four SATA hard drives. They’re going to find a home in a 1U server that I’m planning to purchase in the next couple of weeks. I ordered the hard drives early because I need one or two large, empty drives so I can make some major partitioning changes to my home file server.

I had some simple requirements for the drives in this new server. They had to be large, reasonably fast, RAID friendly, and they needed to fit in the 1U chassis. It would have been nice to be able to use some server-grade 10K or 15K RPM disks, but they just don’t have the capacity I need, and they are a quite a bit more expensive. The best fit for this server are 2 TB or 3 TB 7200 RPM desktop drives.

In the distant past

In the latter days of IDE hard drives, I almost exclusively used drives manufactured by Western Digital. I had reasonable luck with them, and their RMA process was very convenient. They didn’t make you jump through hoops to prove that you had a bad drive, and they would cross ship your replacement drive for free.

I once had a drive in my home desktop machine’s RAID 5 fail, and I picked up a Maxtor 160 GB IDE drive as an emergency replacement. The drive worked fine for a few months, but then it started dropping out of the RAID due to read and write errors.

Maxtor wouldn’t RMA the drive unless I ran their diagnostic software. Since the drive was plugged into a 3ware RAID controller, their software couldn’t see the drive. I had to move it to one of the motherboard’s IDE ports. Their software miraculously claimed to have fixed the drive. I had to repeat this process three more times over the next several months before it finally admitted the drive was bad. This made me really start to appreciate Western Digital’s RMA policy.

Early Western Digital SATA drives were also quite handy, since they had both SATA and IDE power pins. In those days, most power supplies only had one or two SATA port connectors, so this was terribly handy.

I continued to use Western Digital’s SATA drives right up until about three or four years ago. I picked up some of their 1 TB, 7200 RPM hard drives to use in a Linux software RAID 10. Those drives liked to randomly drop out of the RAID due to read timeouts, which seems to still be a common problem with many of Western Digital’s drives.

Switching to Seagate

I replaced those 1 TB Western Digital drives with equivalent models from Seagate, and I’m still using most of those drives today. I did have to RMA one of them, though. The process isn’t quite a slick as Western Digital’s used to be, but it wasn’t insane like Maxtor’s either. Seagate’s drives also came with five-year warranties, which beat Western Digital by two years.

Things are looking pretty ridiculous this year

Seagate doesn’t seem to have any drives with a five-year warranty anymore. The drives in the capacities and speed that I was shopping for, they were all one-year or two-year warranties. Here’s where it seems to get stupid: drives with the same model number can have either a one-year or two-year warranty. They may even have a different number of platters, heads, and entirely different performance characteristics.

In the future

I did buy four Seagate drives today, but I’m not sure I will be buying any more drives from Seagate in the near future. I’m not too enthusiastic about the shorter warranty periods. I have four or five 1 TB Seagate drives in the room with me here, and many of them will still be under warranty after the warranty on these new drives expire.

I’m even less happy about that fact that I have no idea what sort of drives are going to arrive at my door later this week. I don’t have any solid data, but the Internet is leading me to believe that I may get some combination of two-platter and three-platter drives. The two-platter drives are slower, and they may not be as well made as the three-platter drives.

It sure feels like a conundrum.

Update: The drives arrived today. If I am decoding the serial numbers correctly, one is the superior 6 head model. The other three are the slower two-platter, four-head model. All four were made in Thailand, and I am pretty sure that is good news.

Update: According to Backblaze’s statistics, Seagate drives may not have been the best choice.

Update: Shortly after writing this entry, I discovered Seafile. I had to scale my plans back. Most of the software I looked at did not scale well to the hundreds of thousands of files in my home directory, especially when some of those files are changing several times each minute. I have been using Seafile for a few weeks, and so far I am very happy with the results.

Almost two months ago, I looked into the idea of replacing Dropbox. No single piece of software was able to meet my needs. Since then, I’ve put some thought into what my goals are. It looks like I will be able to meet the majority of my requirements by combining two or three different pieces of software.

The goals

I want to synchronize the contents of my 25 GB home directory between multiple locations. I would like to be able to replace my existing backup plan with one of these synced copies. That means at least one location has to have file versioning or snapshotting. If I want to be able to eliminate my existing backup plan, at least one of these copies has to be stored securely off-site.

I need a convenient way to share files. I can easily share entire directories using Dropbox, even if the recipient only has a web browser.

I am most definitely addicted to Dropbox’s Android app. Automatically uploading photos is extremely handy. I will most definitely need to be able to replicate this functionality.

They all fall short of the goals

I could most likely meet all of these goals using Dropbox encrypts your data on the server side, and they have all the keys needed to look at your data.

SparkleShare and ownCloud can each meet the first two goals, but not without problems. You can encrypt your SparkleShare repository on the client-side, but then you won’t be able to share files. ownCloud has the automatic photo uploading covered on Android, but there are no plans to add client-side encryption to ownCloud.

BitTorrent Sync can easily and securely move my data around, but the data it moves around is stored unencrypted on each end. That means that you have to put a lot of trust in each endpoint.

My actual plans

I should be able to accomplish most of my goals with a combination of BitTorrent Sync and a snapshot-capable file system, like ZFS or btrfs.

Sync and back up my home directory

To start things off, I plan to synchronize my laptop’s entire home directory with my home file server. This will require some effort, and it will be a bit of a challenge. My file server does not currently have the capability of taking snapshots, and I don’t power it up 24/7 anymore.

I plan to have the server wake up long enough to sync up with my laptop once each day. After the syncing is finished, the server will record a snapshot and shut itself down. The hard drives will only have to spin for a few minutes each day, and it will still be easy to keep my home office cool during the upcoming Texas summer.

Around seven years ago, I replaced a small laptop and a bulky desktop computer with just a single laptop. I haven’t had a desktop computer since then, and I don’t know if I ever will again. Late last year, my friend Brian very nearly convinced me to build a desktop computer for myself. Having good, solid sync capability for my entire home directory would make it much easier to revert back to using two devices again.

Sharing files

I had originally planned to use ownCloud for this, but I thought of a simpler solution. I am going to create a directory called “Public” on my laptop, and I will use BitTorrent Sync to push it to a web server. That should cover nearly all of my random file sharing needs.

BitTorrent Sync has its own built-in, secure file-sharing system, but the recipient needs to have the BitTorrent Sync client installed in order to receive the files. This option isn’t ideal, but I wouldn’t be surprised if it ends up coming in handy every now and again.

Camera uploads

This is the other reason I had originally planned on supplementing BitTorrent Sync with ownCloud. The folks over at BitTorrent Labs had an ARM build of their client available since they launched, and there’s already an app in the Google Play store to install it for you.

I can probably get away with syncing the photos directly to a web server. The Titanium Backup backups of my phone and tablet are probably using most of my Dropbox and just sync that backup elsewhere.

Something is missing

That something is the off-site backup. I don’t have an ideal solution for this part of the problem yet. I will leave my existing rdiff-backup and Duplicity backups in place until I figure out a better plan.

What is the first step?

I will finally be home, in my own office, sitting in my own chair at my own desk in just under one week. When I get there, one of the first things I’ll have to do is find some spare disks to temporarily hold all the data on my file server. Then I can tear down the existing RAID 6 array, and replace it with a ZFS RAID-Z6 volume.

That is actually the hardest part. The rest should be easy, so stay tuned!

/2013/04/self-hosted-cloud-storage-solution-owncloud-vs-sparkleshare.html “Self-Hosted Cloud Storage Solutions: ownCloud vs. SparkleShare vs. BitTorrent Sync vs. Seafile”