[Kde-pim] Data loss: kmail2 must not use existing [Folder-xy] settings on random new folders

[Volker Krause]

On Thursday 15 December 2011 11:53:06 Sven Burmeister wrote:
> Am Donnerstag, 15. Dezember 2011, 09:27:04 schrieb Volker Krause:
> > The uniqueness is guaranteed by the database, if you want ensure
> > uniqueness
> > across database loss you essentially start to create a second database for
> > information that must not be lost, bringing us back to square one.
> 
> No. Creating a real unique id would minimise the chances of re-occurrence,
> so in theory, even if you would create a million databases no id would
> re-appear. No need for a second database.

Right, that would however take away the range-compression optimizations in the 
protocol and the database queries to a large extend (which rely on a high 
probability of sequential ids).

> But as already suggested, this is not even needed if there is only a single
> real unique id, i.e. as unique as message ids are. Any client can store that
> id the first time it accesses akonadi or even prefix its configs with it
> and compare it on each following re-connect.

Something like UIDVALIDITY in IMAP, yes that would work. However, it requires 
persistent storage that the user does not randomly delete. So far we assume 
that the database, but that's apparently not good enough.

> So in a way akonadi would supply a unique prefix to all its pseudo-unique
> ids.
> > That's where the real bug is, Akonadi tries too hard to recover from a
> > catastrophic error (loss of database), without telling you to reset your
> > setup.
> 
> Which is why I suggested to create a real unique id when creating akonadi's
> database in order to allow any client to check whether the database changed.
> Store that unique-id in the database and as soon as the database is
> replaced the unique-id will change as well.
> 
> This does not make it as robust as it would be with real unique ids, i.e. a
> user can still empty akonadi's db manually – but that really is a croner
> case IMHO.

So manipulating the database is a corner case, deleting it isn't? Doesn't make 
sense to me, the only way you should interact with that database is through 
Akonadi, everything else is not supported and should be detected as an error 
(that part needs to be improved, obviously).

> > No, it is a catastrophic error scenario.
> 
> You might not like to hear this but if akonadi suggests on wikis etc. that
> killing its db will not lose any email, erasing the database will be seen as
> ok and used as soon as something is not working, i.e. to check whether it
> got corrupted. People do the same with nepomuk or amarok databases. IMHO it
> does not make sense to discuss how wrong this is, but it does make sense to
> avoid any data loss even in that case. 

Deleting random data as a diagnostic method? Come on, if that's really what 
our wikis are suggesting, then that's the high priority bug we need to address 
ASAP.

If there are real use-cases we need to provide real solution for those. "It's 
too big" has been addressed by several changes for 1.7/4.8, and further 
optimizations exist as unfinished work branches. "Is it corrupted?" will also 
get some new diagnostics in 1.7, but can probably use some more work. Those 
are the only reasons I saw in this thread so far, what else do you have in 
mind?

> Even more so if it is as easy as
> creating one single unique id and making it available to clients.

I am very reluctant to add extra complexity for the common use case just to 
protect people that insist on doing something you are not supposed to be 
doing. While your suggestion is conceptionaly simple, the implementation would 
still have to touch a lot of code throughout the entire stack, and worse, it 
would require some kind of config migration.

Adding extra safety checks for invalid/corrupt setups makes sense though, and 
has far less impact on the general stability I think.
 
> > Nevertheless, the already suggested approach of moving the folder settings
> > from kmailrc into Akonadi collection properties makes sense anyway, for
> > various other reasons as well.
> 
> It does not solve all problems though. Filters would e.g. still point to the
> wrong folder which might be the trash being emptied on kmail close or some
> public imap folder. And there might be more scenarios I did not yet come
> across. All of which can be solved by adding a single unique ID to
> akonadi's database and making it available to clients for comparison.

Sure, the part I'm most worried about in this scenario are the change replay 
queues anyway, filters and expiry are harmless corner cases compared to that.

regards
Volker
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 190 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-pim/attachments/20111215/9f8a3216/attachment.sig>
-------------- next part --------------
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/