Outsourcing My Self-Hosted Cloud Storage

| Comments

Five years ago, I was on the hunt for a Dropbox replacement. I wanted something that was open source, and something that I could host myself. The solution needed to be scalable There are a lot of options with varying feature sets.

I wrote a few paragraphs about each project as I tested them. I got farther with some solutions than others. Some packages didn’t have client-side encryption, so they were immediately scratched off my list. Some slowed to a crawl when I tried to sync more than a few thousand files. I needed to sync at least 100,000 files, so they weren’t going to work.

Once my testing reached Seafile, I never looked back. Seafile lets you break your files up into groups called libraries. In 2013, each Seafile library could easily scale up to 20,000 files. That was just enough to manage the largest folders in my home directory, and their efficiency has improved greatly over the last five years.

Catastrophe strikes!

I’ve used a colocated server for all my hosting needs for nearly a decade. The machine I’ve been using for the last four or five years is a 16-core Xeon with 48 gigabytes of RAM and four 2 TB hard drives. It had virtual machines hosting several websites, some random game servers, a Zimbra email server, DNS servers, and my Seafile server.

Late last year, my aging server finally died. The most important services running on that machine are the web servers that host my blog and Brian’s blog. We were smart enough to start using a CDN a few years ago. Cloudflare had the majority of our pages in cache, so we weren’t going to experience a serious interruption there.

Without the web servers, we wouldn’t be able to publish new content, so we both wanted to get back up and running pretty quickly. I have an extra 1U server here in my home office. I needed to decide whether I should get that up, running, and ready to go. Would it be cheaper and easier to migrate to the cloud?

It is definitely easier and faster, but my initial math didn’t make it all that much cheaper. All the reputable email provides charge by the user, and my storage needs are fairly high. My quick math put me within $10 or $20 of what I was already paying for my colo. I already own the hardware, so I wasn’t worried about that cost. This is a topic for another blog post!

Where should I host Seafile?

For my web hosting, it was a toss-up between Linode and DigitalOcean. I went with DigitalOcean, because they had better pricing on additional storage. I would need at least 350 GB just for my Seafile server’s data—that includes my data and my wife’s data.

I quickly set up Nginx on a $12 DigitalOcean VPS, and it has been happily hosting our websites ever since. Once that was done, I started investigating managed Seafile services. There aren’t a lot to choose from, and most of them are a lot more expensive than Dropbox or Google Drive.

I found a host that was cheap enough to make me nervous. Prometeus.net offers managed Seafile hosting on servers in Romania starting at around $30 per year. That gets you 200 GB of storage. I had to go with the 400 GB plan for just under $50 per year.

Adding 400 GB of storage to my DigitalOcean VPS wouldn’t cost me $50 per month, but it wasn’t all that far off! Not having to maintain a Seafile server instance has value, too, so I figured Prometeus was worth a shot!

Is cheap Seafile hosting slow?

Prometeus’s Seafile hosting is definitely on the slow side. I’m used to the speeds I get when syncing to my colocated server in downtown Dallas. It was only a few hops away, and it had no trouble maxing out my 100 megabit FiOS link. I could easily upload a gigabyte of data every minute or two.

UPDATE 2019-05: We uploaded my wife’s data to a Seafile server at Prometeus.net this month. We synced 100 GB of data in less than five hours. It probably uploaded much faster than that, but that’s about how long I walked away for!

Syncing my 320 GB of data to Romania was glacially slow. It took almost a solid week to upload all my data. Once my data got up there, it wasn’t so bad. It has been a long time since I used Dropbox, but they never managed to max out my FiOS link, either!

My Seafile Libraries

Seafile has me covered in two valuable ways. It keeps the files on my desktop, laptop, and tablet in sync. Most importantly for me, that means my blog posts can be edited from anywhere. If I save this blog post right now and open my laptop, it will be right where I left off. Blog posts are tiny, so they still sync almost instantly. Raw photos from my DSLR are a bit slower to sync, but they still usually only take a few minutes.

I also rely on Seafile for my backups. Seafile’s synchronization gives me three local backups of the most current state of my data. The server provides me with 90 days of history.

I feel like that has me covered pretty well. If a hard drive fails, I have local backups. If a file gets damaged, overwritten, or deleted, I can get it back from the server. If the cheap server disappears, I still have three local copies of my data.

Relying on Seafile for backups has been amazing. I don’t miss the old days of daily or hourly backups. All my changes are backed up in 20 or 30 seconds. Not only are my changes backed up, but they’re synced to at least one other machine almost instantly. How can it get any better?

Can you trust these guys with your sensitive data?

When you create a Seafile library, you can choose to encrypt it. If you do encrypt your libraries, all encryption and decryption is done on your end. The server doesn’t have access to your encryption keys.

You should encrypt your libraries. If you do, the folks at Prometeus don’t have access to your data. Seafile is open source, so it is possible to verify that the keys aren’t transferred to the server. I’m using the Seafile client from the Ubuntu repositories, so I’m trusting that the folks at Canonical aren’t sneaking anything into the binary.

I’m not worried about Prometeus stealing my data. I don’t have anything all that sensitive anyway.

Does Seafile’s deduplication work with encryption?

The short answer is yes. The truth is slightly more complicated.

Seafile slices your file up into pieces before uploading them to the server. Each block is hashed, and if the server is already storing a block with the same hash, then the client doesn’t need to upload the block. This saves time, bandwidth, and space on the server.

When encryption isn’t enabled, I believe Seafile deduplicate between any library stored on the server. If you share the server with your friends and family, and you all have a copy of the same large video, only one of you will have to upload it and only one copy of the data will be stored on the server.

If you’re using encrypted libraries, then the Seafile server can only deduplicate within the confines of a single library. This is a limitation that I am happy to live with, because I encrypt most of my libraries.

Why not just use Dropbox?

I don’t trust Dropbox. They hold the keys to access your data. This is true of Dropbox and many of their competitors. Some of Dropbox’s competitors claim that they encrypt your data on the client, and they have no way to access your data on their end. Unfortunately, their client software is usually closed source, so you have no way to verify whether or not they are doing things correctly.

Seafile is open source. If I had any idea what I was doing, I could look at the code and make sure it isn’t doing anything scary. I don’t have any idea what I’m doing, but my hope is that someone else does. This isn’t a fool-proof plan, but I’d rather put my faith in the open-source community than a profit-driven company.

My friend Brian has been using CrashPlan, but they recently shut down their consumer-facing business. That leaves him searching for a comparable service. I’ve been using Seafile for quite a few years, and I expect to be using it well into the future.

If the business entities behind Seafile wind up changing direction, the software won’t have a definitive drop-dead date. I’ll expect to be able to continue on doing what I’m doing until the software gets old, rickety, and completely abandoned—I will be able to move along on my own schedule.

What if the Seafile hosting provider disappears?

That will be a bummer. I’ve already gotten more than $50 of value out of the service. In the worst case, I’ll just have to set up my own Seafile server again. I’m not worried about having to do that, but it would be much better if I don’t have to!

Conclusion

Prometeus’s Seafile hosting is an excellent value. The speeds are reasonable, and the storage is affordable. If you’re already running your own Seafile server, and you are looking to move to a hosted solution, I’d say go for it. Upgrading Seafile on your own server is easy. Running garbage collection is easy. Even so, I’m excited about not having to worry about doing either one again.

Comments