Cloud Backup Software

Status
Not open for further replies.

Linwood Ferguson

Linwood Ferguson
Lightroom Guru
Joined
Jan 18, 2009
Messages
2,587
Location
Fort Myers, FL
Lightroom Experience
Advanced
Lightroom Version
Classic
I have been spending quite a bit of time over recent weeks looking into my cloud backups, and want to see if there are any out there I may have missed that might meet my goals.

First, a caveat - I want software that can back up to multiple providers, NOT software provided by the vendor of the storage. I believe that storage vendors, especially those who offer unlimited backups, exercise too much control over what and how things are backed up. I want an independent program I control.

As a simple example: Backblaze offers $5/mo unlimited backups, but limits you to 30 day retention on prior versions, and does not back up lots of things (for example, it does not back up a .ISO file, for reasons lost on me). The latter is changeable if you notice; the former is not.

On the other hand, Backblaze B2 is a la carte storage, and if you use it, you can control entirely what you store there and how. The break even point is about 1TB of storage, above that you pay a bit more for control, below that you actually pay less to have your own software.

So...my goals:

1) Versioned (point in time) backups completely under my control (including automatic runs)

2) Optional encryption as it leaves my system, NOT in-vendor encryption; storage vendor must not have keys

3) Fast, local backups to EHD or NAS or other computers with similar features (cloud backups are my backup-of-last-resort, always have local ones).

4) A variety of mechanisms to ensure reliability - consistency checks, verify-after-write or verify-later, preferably capitalizing on storage features (e.g. S3 provides MD5 without reading a file; B2 provides SH1 without reading a file, which could be used to compare to live on-disk copies).

5) Some track record and indications of stability and support going forward.

6) Broad cloud vendor support but in particular three I've looked at are S3 (and Glacier), Amazon Cloud Drive, B2 as they have the most attractive pricing.

The two I am landing on are:

Cloudberry Online Backup - from Cloudberry Labs. Has by far the most broad list of supported storage, and is definitively a backup product (as opposed to sync), with point in time, easy restores, incremental and differential backups (including large file backups of only changed blocks), encryption... it really is by far the most complete, except for these issues:

- Mediocre at best support (and it is paid, albeit cheap), and mostly no support. To be fair, the product just works most of the time (I have used it for years).

- The cheap ($30) version only will support up to 1TB total size, for unlimited you buy their enterprise at $300. So for most photographers it's a $300 solution not $30 (I bought before this rule so am grandfathered).

- The consistency check and MD5 verification features are poorly documented, consistency check seems to work fine, verification by MD5 I have no clue and have not been able to find out.

Goodsync - from Sieber Systems has been around for a long time, but is (IMO) poorly positioned from a marketing standpoint, as it is strongly a "sync" tool, that happens to do backup. In fact it does a very fine job of backup, but it is hard to tell if you are just browsing. It also requires you jump through a few hoops to avoid the sync aspect and concentrate on backup. But once you do, most the features are there - prior versions (though awkward to retrieve), excellent consistency and verification checks, no size or usage limits. They also have a peer component if you control the other side of the connection (e.g. another computer or NAS) that let's the backups run extremely efficiently, I can actually move at wire speed on 1gbs ethernet. They also produce very frequent updates with bug fixes and features, it is very actively developed and has been for a long track record. I was attracted to them as they were the only vendor who did a good job of Amazon Cloud Drive. Their downsides:

- Not very broad cloud support, for example no B2 (nor plans)
- Well documented but arcane features for backup; e.g. point in time restores are purely manual
- Support is fast and responsive but moody and not very helpful

Most other products I have discounted. For example, I tested ARQ but its Cloud Drive just did not work properly, and when questioned on consistency check or verification processes, was basically told "trust the cloud" when I was offering examples of where the backup in the cloud was corrupted. That ended that trial.

A lot of tools (e.g. oDrive) were very cool but pure sync, no backup support (e.g. point in time restores).

I ignored tools locked to their own cloud (Acronis is one of the better known).

Cloudbacko (which is very much a Cloudberry look-alike) didn't support Glacier or B2, so it did not offer the price of storage advantage.

A ton of others that do not come to mind now, but generally lacked some killer feature, especially in the "cheap but good" storage vendor areas of Glacier, B2, ACD.

So.... my search is never over really. I'm posting to see if others have found a better solution, or other products I should look at.

Or if you are looking, you may want to consider these.

By the way, for reference, as of this writing cloud storage prices to some extent are summarized here though it omits Glacier ($0.007 but with very expensive downloads) and Amazon Cloud Drive (unlimited at $60/mo).
 
Have you looked at CrashPlan? I know that they offer several different encryption options including your own private key.

-louie
Breaks the OP's requirement of storage independence.

If you want hardcore backup which is storage independent, I have used the following:
Amanda Network Backup: Open Source Backup for Linux, Windows, UNIX and OS X
Areca Backup - Official Website
Bacula | Open Source Backup, Enterprise ready, Network Backup Tool for Linux, Unix, Mac, and Windows

Personally I like Bacula the best of them. But all are very good, but you need to determine which one will support the kind of remote storage services you desire.
 
Have you looked at CrashPlan? I know that they offer several different encryption options including your own private key.

Thanks, Louie, as tspear noted I was avoiding those. It is perhaps worth noting why, as my rationale is a bit dated, though I think still relevant.

Back in the early days, I tried Mozy which at the time offered unlimited storage as well. I do not have excessive needs, at the time I suspect it was 500GB or so. Mozy was unlimited.

Well, they were unlimited much like AT&T's variations of unlimited data plans were -- once you got to a certain size, they progressively slowed your upload more and more until you effectively could not upload at all.

But more significantly they simply lied about this. You could open a ticket, and they had a whole script about how it was your network, your computer... always "your". Various people (including me) spent time sniffing the links and could see definitively it was them throttling, and eventually they came clean and if I recall just removed the unlimited option and are now one of the pricier plans (125GB is $10/mo).

They were not the only one, most of the unlimited or nearly unlimited plans at the time had some technique to make sure their "all you can eat" plans did not apply to those with large appetites, whether they would just cancel your plan, throttle you, or take control over what was backed up.

As the price of storage went down, some plans shifted to a more literal "unlimited" and it is quite possible some plans, including crashplan, are now truly not restricted. But two of the most frequent issues is WHAT they back up, and how long old file versions (included deleted files) are kept. I looked briefly at Crashplan this morning and they seem quite reasonable in these settings, at least what I could see. Others are not so much (my example above of Backblaze's unlimited plan only keeping old versions 30 days). The aggressiveness with which some of these plans try to limit what is backed up can be hidden issues as well, for example on most if you happen to run the program without a EHD plugged in (that was before, and was backed up), it will think all the files are deleted and mark them for removal. If on Backblaze, they would vanish in 30 days. Crashplan let's you change that time line, but only if you remember to change it. And there are reports of Crashplan doing excessive (i.e. slow) deduplication for wan vs lan backups as a way to minimize their storage cost at your upload expense (I do not know this to be true, just did some reading this morning).

Fundamentally (and perhaps becoming less appropriate in the race to zero cost storage) I look at incentives. Vendors providing unlimited storage have incentives to limit your use. Vendors that you pay per gigabyte do not (in fact the reverse). While perhaps most vendors today are on the up-and-up, and space so cheap they just suck it up for big consumers -- I have a basic mistrust of companies when their financial incentives are not aligned with my needs.

Breaks the OP's requirement of storage independence.

If you want hardcore backup which is storage independent, I have used the following:
Amanda Network Backup: Open Source Backup for Linux, Windows, UNIX and OS X
Areca Backup - Official Website
Bacula | Open Source Backup, Enterprise ready, Network Backup Tool for Linux, Unix, Mac, and Windows

Personally I like Bacula the best of them. But all are very good, but you need to determine which one will support the kind of remote storage services you desire.

Thank you. I'll look. Got a lot of space time while I'm doing the initial upload to B2, about 300gb into about 1.5TB. :(
 
I do not see that any of these support any cloud backups. The first had a component called zmanda for cloud backup, but now that link takes you to Carbonite and they appear to have abandoned it.

These now appear to be computer to local storage, or at best FTP (like) storage, most clouds use some variation of http for uploads.

Unless I'm missing something?
 
I've used two cloud service plans, both are unlimited. The first that I tried was Carbonite. It worked well until I wanted to back up data un multiple disks. This option was available in Windows but not OS X. The I switched to CrashPlan. One problem with cloud backup services is durability. There is always the possibility that the companies unlimited plan is that it may not be financially sound The Company may go under, sometimes without notice For this reason, I have both a local backup and a cloud backup. Local backup is faster but at the risk of local disaster wiping out every copy of your data. CrashPlan offers both a local and a cloud service. the Local backup is free.

Crash plan meets my needs though they may not meet those of Linwood
 
Cletus, I've been reading more about crashplan this morning and it does seem to address many of the issues I had with other vendor plans. I still have that nagging hesitation about "unlimited" plans though, despite realizing as storage gets cheap their motivation to rig the process to slow you down is reduced. But I like what they say about their product.

To me cloud backup is the "my computer and all local copies are gone, last ditch" recovery location, as you mention. I figure the chances of that vendor going under the very week my house burns up (floats away, whatever) is pretty small. It's what long, long ago I would keep on a tape in a bank safety deposit vault (and wonder if when the flood came it was water proof -- not a weird question in flat coastal florida).

Though, frankly, I may find over time that something like crashplan is really the right answer now, and I'm solving a problem from the past.
 
I do not see that any of these support any cloud backups. The first had a component called zmanda for cloud backup, but now that link takes you to Carbonite and they appear to have abandoned it.

These now appear to be computer to local storage, or at best FTP (like) storage, most clouds use some variation of http for uploads.

Unless I'm missing something?

Nope. You then need to get a way to mount it. Or perform a local backup and then sync backup to the remote cloud....
Like I said, you would be going hardcore that route.
 
Cletus, I've been reading more about crashplan this morning and it does seem to address many of the issues I had with other vendor plans....
To me cloud backup is the "my computer and all local copies are gone, last ditch" recovery location, as you mention. I figure the chances of that vendor going under the very week my house burns up (floats away, whatever) is pretty small. It's what long, long ago I would keep on a tape in a bank safety deposit vault (and wonder if when the flood came it was water proof -- not a weird question in flat coastal florida)....
I came to the conclusion that Crashplan meets my needs. My thinking is that CrashPlan might eventually abandoned the "Unlimited" plan, but I will have ample time to do something about it. And if the company does go under, I also think I'll have ample time to finds another solution.

I'm sitting on the other side of the GoM about 850 miles west of you with a judicious eye on the weather in August and September. CrashPlan is my ultimate disaster recovery plan. Knowing that they will send my a physical drive with all of my backed up data makes the cloud plan all the more attractive. I used to work in IT and have dealt with disaster recovery plans at the corporate level. You can over think it but you can never have too many back ups.
 
You can over think it but you can never have too many back ups.

I've worked for a company whose data center was in the middle of the damage from the Northridge Earthquake in LA, and later for a company where Wilma hit their west coast florida data center, crossed the state strengthening as it went and hit their east coast data center. We were lucky (and a little well prepared) in all three cases.

Mother nature is creative; planning ahead is a goodness. The same storm that hit in Florida (Wilma) was forecast to bring as much as 14' storm surge to my house. It didn't -- but that would have been about 2' above the top of my monitor.
 
I have been spending quite a bit of time over recent weeks looking into my cloud backups, and want to see if there are any out there I may have missed that might meet my goals.

First, a caveat - I want software that can back up to multiple providers, NOT software provided by the vendor of the storage. I believe that storage vendors, especially those who offer unlimited backups, exercise too much control over what and how things are backed up. I want an independent program I control.

As a simple example: Backblaze offers $5/mo unlimited backups, but limits you to 30 day retention on prior versions, and does not back up lots of things (for example, it does not back up a .ISO file, for reasons lost on me). The latter is changeable if you notice; the former is not.

On the other hand, Backblaze B2 is a la carte storage, and if you use it, you can control entirely what you store there and how. The break even point is about 1TB of storage, above that you pay a bit more for control, below that you actually pay less to have your own software.

So...my goals:

1) Versioned (point in time) backups completely under my control (including automatic runs)

2) Optional encryption as it leaves my system, NOT in-vendor encryption; storage vendor must not have keys

3) Fast, local backups to EHD or NAS or other computers with similar features (cloud backups are my backup-of-last-resort, always have local ones).

4) A variety of mechanisms to ensure reliability - consistency checks, verify-after-write or verify-later, preferably capitalizing on storage features (e.g. S3 provides MD5 without reading a file; B2 provides SH1 without reading a file, which could be used to compare to live on-disk copies).

5) Some track record and indications of stability and support going forward.

6) Broad cloud vendor support but in particular three I've looked at are S3 (and Glacier), Amazon Cloud Drive, B2 as they have the most attractive pricing.

The two I am landing on are:

Cloudberry Online Backup - from Cloudberry Labs. Has by far the most broad list of supported storage, and is definitively a backup product (as opposed to sync), with point in time, easy restores, incremental and differential backups (including large file backups of only changed blocks), encryption... it really is by far the most complete, except for these issues:

- Mediocre at best support (and it is paid, albeit cheap), and mostly no support. To be fair, the product just works most of the time (I have used it for years).

- The cheap ($30) version only will support up to 1TB total size, for unlimited you buy their enterprise at $300. So for most photographers it's a $300 solution not $30 (I bought before this rule so am grandfathered).

- The consistency check and MD5 verification features are poorly documented, consistency check seems to work fine, verification by MD5 I have no clue and have not been able to find out.

Goodsync - from Sieber Systems has been around for a long time, but is (IMO) poorly positioned from a marketing standpoint, as it is strongly a "sync" tool, that happens to do backup. In fact it does a very fine job of backup, but it is hard to tell if you are just browsing. It also requires you jump through a few hoops to avoid the sync aspect and concentrate on backup. But once you do, most the features are there - prior versions (though awkward to retrieve), excellent consistency and verification checks, no size or usage limits. They also have a peer component if you control the other side of the connection (e.g. another computer or NAS) that let's the backups run extremely efficiently, I can actually move at wire speed on 1gbs ethernet. They also produce very frequent updates with bug fixes and features, it is very actively developed and has been for a long track record. I was attracted to them as they were the only vendor who did a good job of Amazon Cloud Drive. Their downsides:

- Not very broad cloud support, for example no B2 (nor plans)
- Well documented but arcane features for backup; e.g. point in time restores are purely manual
- Support is fast and responsive but moody and not very helpful

Most other products I have discounted. For example, I tested ARQ but its Cloud Drive just did not work properly, and when questioned on consistency check or verification processes, was basically told "trust the cloud" when I was offering examples of where the backup in the cloud was corrupted. That ended that trial.

A lot of tools (e.g. oDrive) were very cool but pure sync, no backup support (e.g. point in time restores).

I ignored tools locked to their own cloud (Acronis is one of the better known).

Cloudbacko (which is very much a Cloudberry look-alike) didn't support Glacier or B2, so it did not offer the price of storage advantage.

A ton of others that do not come to mind now, but generally lacked some killer feature, especially in the "cheap but good" storage vendor areas of Glacier, B2, ACD.

So.... my search is never over really. I'm posting to see if others have found a better solution, or other products I should look at.

Or if you are looking, you may want to consider these.

By the way, for reference, as of this writing cloud storage prices to some extent are summarized here though it omits Glacier ($0.007 but with very expensive downloads) and Amazon Cloud Drive (unlimited at $60/mo).
I have been spending quite a bit of time over recent weeks looking into my cloud backups, and want to see if there are any out there I may have missed that might meet my goals.

First, a caveat - I want software that can back up to multiple providers, NOT software provided by the vendor of the storage. I believe that storage vendors, especially those who offer unlimited backups, exercise too much control over what and how things are backed up. I want an independent program I control.

As a simple example: Backblaze offers $5/mo unlimited backups, but limits you to 30 day retention on prior versions, and does not back up lots of things (for example, it does not back up a .ISO file, for reasons lost on me). The latter is changeable if you notice; the former is not.

On the other hand, Backblaze B2 is a la carte storage, and if you use it, you can control entirely what you store there and how. The break even point is about 1TB of storage, above that you pay a bit more for control, below that you actually pay less to have your own software.

So...my goals:

1) Versioned (point in time) backups completely under my control (including automatic runs)

2) Optional encryption as it leaves my system, NOT in-vendor encryption; storage vendor must not have keys

3) Fast, local backups to EHD or NAS or other computers with similar features (cloud backups are my backup-of-last-resort, always have local ones).

4) A variety of mechanisms to ensure reliability - consistency checks, verify-after-write or verify-later, preferably capitalizing on storage features (e.g. S3 provides MD5 without reading a file; B2 provides SH1 without reading a file, which could be used to compare to live on-disk copies).

5) Some track record and indications of stability and support going forward.

6) Broad cloud vendor support but in particular three I've looked at are S3 (and Glacier), Amazon Cloud Drive, B2 as they have the most attractive pricing.

The two I am landing on are:

Cloudberry Online Backup - from Cloudberry Labs. Has by far the most broad list of supported storage, and is definitively a backup product (as opposed to sync), with point in time, easy restores, incremental and differential backups (including large file backups of only changed blocks), encryption... it really is by far the most complete, except for these issues:

- Mediocre at best support (and it is paid, albeit cheap), and mostly no support. To be fair, the product just works most of the time (I have used it for years).

- The cheap ($30) version only will support up to 1TB total size, for unlimited you buy their enterprise at $300. So for most photographers it's a $300 solution not $30 (I bought before this rule so am grandfathered).

- The consistency check and MD5 verification features are poorly documented, consistency check seems to work fine, verification by MD5 I have no clue and have not been able to find out.

Goodsync - from Sieber Systems has been around for a long time, but is (IMO) poorly positioned from a marketing standpoint, as it is strongly a "sync" tool, that happens to do backup. In fact it does a very fine job of backup, but it is hard to tell if you are just browsing. It also requires you jump through a few hoops to avoid the sync aspect and concentrate on backup. But once you do, most the features are there - prior versions (though awkward to retrieve), excellent consistency and verification checks, no size or usage limits. They also have a peer component if you control the other side of the connection (e.g. another computer or NAS) that let's the backups run extremely efficiently, I can actually move at wire speed on 1gbs ethernet. They also produce very frequent updates with bug fixes and features, it is very actively developed and has been for a long track record. I was attracted to them as they were the only vendor who did a good job of Amazon Cloud Drive. Their downsides:

- Not very broad cloud support, for example no B2 (nor plans)
- Well documented but arcane features for backup; e.g. point in time restores are purely manual
- Support is fast and responsive but moody and not very helpful

Most other products I have discounted. For example, I tested ARQ but its Cloud Drive just did not work properly, and when questioned on consistency check or verification processes, was basically told "trust the cloud" when I was offering examples of where the backup in the cloud was corrupted. That ended that trial.

A lot of tools (e.g. oDrive) were very cool but pure sync, no backup support (e.g. point in time restores).

I ignored tools locked to their own cloud (Acronis is one of the better known).

Cloudbacko (which is very much a Cloudberry look-alike) didn't support Glacier or B2, so it did not offer the price of storage advantage.

A ton of others that do not come to mind now, but generally lacked some killer feature, especially in the "cheap but good" storage vendor areas of Glacier, B2, ACD.

So.... my search is never over really. I'm posting to see if others have found a better solution, or other products I should look at.

Or if you are looking, you may want to consider these.

By the way, for reference, as of this writing cloud storage prices to some extent are summarized here though it omits Glacier ($0.007 but with very expensive downloads) and Amazon Cloud Drive (unlimited at $60/mo).

I like the idea of BackBlaze B2 as a cloud storage solution at low cost. Then we need to get backup software what about Duplicati an open source project which I am testing out at the moment with BackBlaze B2 as cloud storage ?
 
Status
Not open for further replies.
Back
Top