Backblazeは容量は無制限だけどファイル数は無制限じゃなかった話

Backblaze

最高の無制限オンラインバックアップおよびクラウドストレージサービス

Backblazeってサービスを2年程使ってます。
$50/年で容量無制限のオンラインバックアップが出来るサービスです。

ちなみに後からソースネクストが3990円/年で国内向けに売り出しましたが、中身は一緒なようで同仕様かと思われます。

バックアップ止まったんですけど

ある日こんなメッセージが出てバックアップが停止され再開も効かなくなりました。

WARNING: Backblaze has stopped working properly on this computer! Your bzfileids.dat file is too large, please contact Backblaze customer support.

原因検討も付かないしplease contactとか言ってるので聞いてみました。

サポートの答え

解答抜粋

There is a very specific file on your disk that is part of Backblaze which has bloated up to be too large. It is called "bzfileids.dat" and is found here:

Mac: /Library/Backblaze.bzpkg/bzdata/bzbackup/bzfileids.dat
Windows Vista/7: C:\ProgramData\Backblaze\bzdata\bzbackup\bzfileids.dat
Windows XP: C:\Documents and Settings\All users.windows\Application data\backblazebzdata\bzbackup\bzfileids.dat

This is a very simple file, it is a mapping from your filenames to a totally unique integer ID that is anonymous that we use to identify your files in the Backblaze datacenter. This means we never know any of your file names, or file contents.

For some reason, your computer wants to backup a few million files and your bzfileids has grown very large (yours is over 1 GB). When bztransmit (the process that runs once per hour) starts up, it reads this bzfileids.dat file into RAM. On a normal machine, this is about 20 MB, but on your machine something has gone haywire and bzfileids has grown far too large.

Now, there are several things that contribute to this file being large, so you can think about how this happened and let us know. We're trying to understand this situation better:

A) If you have ever renamed a Time Machine folder at the top of your hard drive, Backblaze will bloat up trying to back it up. It is absolutely not supported to "back up a back up" and Backblaze can only function properly backing up the originals.

B) Lots of files. If you knew of a folder with hundreds of thousands of small files that didn't change much you could back them up differently or exclude them from Backblaze backups.

C) Renaming top level folders with a lot of files. For example, if your top level folder name is "/my_music" and it contains 100,000 file names in it, then when you rename it "/my_great_music" Backblaze needs to add all of those filenames to that bzfileids.dat file which bloats it up. So the best thing you can do is keep your enormous folders the same over a long period.

D) Shorter Path Names. It would be best if your hundreds of thousands of files are on a disk called "d" instead of "disk_that_contains_files" and the top level folder is called "f" instead of "folder_for_lots_of_files". Etc. The shorter the paths, the smaller the bzfileids.dat file is.

It's possible to shrink the bzfileids.dat in case they've been temporarily bloated by one of the above situations, however it requires reuploading all data to the Backblaze servers. You can follow these steps to do that:
1. Visit https://secure.backblaze.com/user_signin.htm and sign in to your Backblaze account with your email address and password.
2. Click on the "Account" link in the upper left hand corner
3. Select your "old" computer from the list of computers.
4. Click the "Delete Computer" link next to it. This will delete the backed up data, the bloated bzfileids.dat and free up the paid license.
5. Click on "Overview"
6. Click the download link for your operating system in the bottom right corner.
7. Install Backblaze.

Here's the problem. If you cannot reduce the number of files or path names significantly, then you absolutely are going to encounter this issue again. We are working on a fix for it, but it is proving to be very difficult and currently our engineering team has no ETA on a fix. If you uninstall and reinstall without changing anything, then your backup might start working again, but you will reencounter this problem a little ways down the road.