[Digikam-users] backup and data integrity

[Jakob Østergaard Hegelund]

On Wednesday 16 January 2008, Arnd Baecker wrote:
> Hi,
>
> one important issue with digital images is the
> question of backup (eg. CD/DVD, optical media, ..
> several separate hard-disks, off-site hard-disks, ...).

I use rsync to an off-site server.

My mother uses rsync to a removable disk.

I work for a company who will be providing affordable remote backup as a 
service to end users.

> Another (maybe often over-looked?) aspect is whether
> the data (both on the master disk and the backups)
> are still correct.
> It may well happen that files just get corrupted
> on the hard-disk.

Sure, that's unlikely but not impossible.

> (I recently had such an experience, where
> fortunately an old back-up on CD allowed to recover
> the few files).
>
> So my question is:
> How do you ensure the correctness of your data?

Use proper hardware with ECC memory.

No overclocked/non-ECC systems.

> What methods are useable and could one maybe
> integrate/provide part of the needed tools
> inside digikam?
>
> One approach might be to use a hash value (eg. md5):
>
> - Digikam could compute a hash value for every
>   image and store it inside of digikam's database.
>
>   This would allow, by an additional tool, to periodically (etc.)
>   check for any possible changes (=corruption) of images.

But the application should not do this.

The reason why we use a modern OS on modern hardware is, that the OS and 
the hardware will work together to provide us with everything from 
hardware abstraction to error correction.

ZFS provides the extra layer of error detection along with error 
correction that you seek. I believe time would be better spent lobbying 
for a port of that technology to Linux (one way or another - yes I am 
aware that there are complications), rather than trying to patch in a 
little bit of that functionality into a single application (which will 
not help the general case).

Anyway, that's my take on the issue. I believe any improvement that 
would be needed, is needed on a general system-wide scale, not on a 
single-application scale.

-- 
Jakob Østergaard Hegelund