Image via

Data Sync Not Invented Here

Posted in Apps, Engineering

You’d think that data syncing is a solved problem for applications. Dropbox has been around for close to 10 years, and they came up with a product model — a local folder whose files (usually) silently merge with upstream servers — that works perfectly fine and is readily available to desktop applications. Hell, there’s Box, Google Drive, Amazon Cloud Drive, Apple iCloud, and Microsoft OneDrive; all of which do pretty much the same thing with files and just feature different tiers of storage space and built-in integrations with other software products by those companies.

Yet, many applications end up developing their own, proprietary syncing mechanisms. I remember my to-do app, Things, spent 3–4 years building their “Things Cloud” so they could assure their users that snippets of text and metadata would persist across iOS and Mac devices. More recently, the journaling app Day One came out with a new version which requires using its own Day One Sync 2.0, supposedly because syncing images in addition to text and metadata is unreliable everywhere else. Then there’s the keyboard utility TextExpander, which went to a subscription model so they can offer — you guessed it — syncing configuration files across devices1.

Are developers just using these syncing features as little more than attempts to lock the user into their applications? As compared to writing out files to known syncing services, it seems like a huge amount of (over)engineering to come up with a custom way to sync data for users, and then have maintain servers for all the years after that initial app purchase. For whatever reason, the reasonable motivations that drove businesses and companies to just pay for AWS and S3 storage don’t seem to apply to syncing services.

As an end-user, my big concern is not just the stability of a privately-run service2, but what happens when the company — and presumably the service — goes under. The good ones may patch their apps to allow third-party syncing, but they may also just take the ball home and shut everything down with no user recourse. For instance, I was concerned enough about the health of Evernote that I ended up moving to a less-polished but more stable competitor.

And while the bigger cloud drive services above aren’t immune to shutdown, they’re in a better spot both because the company can survive much longer than a small indie shop or startup, but more importantly because they also tend to have other company-branded apps built on top. These services also have enough users that discontinuing them would prompt a transition plan from the company itself, if not from end-users and fans of the service directly3. Having these apps use one of a handful of major services also allows the end user to consolidate their sync-able storage accounts, particularly if they’re paying for extra capacity4.

Sadly, it looks more and more like proprietary data syncing is a sellable feature, both in pushing for version upgrades as well as justifying a monthly subscription. I guess as long as the data is fully exportable, I can develop reoccurring backup and even exit strategies in case the app gets discontinued; it’d be a lot more transparent if I can just manipulate the files directly in my cloud provider of choice.

  1. And more than one user is unhappy about the new business model.

  2. That said, I was burned by this pretty recently. When Day One 2 launched and migrated everyone over to its own sync, there was a bug that triple copied every journal entry. It took me 3 hours to dedup. my data before they patched in a fix.

  3. See the ecosystem around RSS readers that flourished with the demise of Google Reader.

  4. I’m looking at you, iCloud and your huge iOS backups.