[Akonadi] [Bug 330895] copy mail folder to other folder on other ressource results in empty mails

[Daniel Vrátil]

https://bugs.kde.org/show_bug.cgi?id=330895

--- Comment #1 from Daniel Vrátil <dvratil at redhat.com> ---
Git commit ff9edf8c0d96b179116558020d5a571a25142795 by Dan Vrátil.
Committed on 05/04/2014 at 21:29.
Pushed by dvratil into branch '1.12'.

Fix potential severe data loss during copy and move operations

Move and copy operations on larger sets of items can take some time, because
we need to make sure that all items have full payload stored in Akonadi. This
is necessary especially with local resources, like maildir, which have cache
timeout set to 5 minutes (after that all their payload is removed from Akonadi
and is fetched from HDD on demand, because it's cheap, fast and does not
unnecessarily duplicate emails in maildir and in database). However fetching
large amount of items via ItemRetriever takes a lot of time, sometimes it can
take even more than 5 minutes. And in such case there is a very high chance,
that the CacheCleaner will just remove the newly cached payloads from Akonadi
again and so when ItemRetriever finishes, many items have empty payload in
Akonadi again. ItemRetriever nor handlers are aware of this howerver, so they
will just make copies or moves of empty items, causing data loss.

This patch introduces CacheCleanerInhibitor, a class which when it is created
will pause the timer in CacheCleaner and resume the timer again when it's
destroyed (so usually when it goes out of scope). Also, this patch adds the
inhibitor to all handlers that use ItemRetriever, so that the the situation
described above does not happen.

The current solution is not perfect because it pauses the entire CacheCleaner
while I think it would be better to be able to temporarily 'blacklist' only
specific collections or items. That would however require much more complex
code and changes, which makes it unsuitable for 1.12 release.

I tried moving 78 000 emails from one maildir to another and all emails were
moved correctly. Move itself has many other problems (CPU, IO, memory, KMail
responsivness, etc.) but that's out of scope of this fix.

M  +49   -0    server/src/cachecleaner.cpp
M  +30   -0    server/src/cachecleaner.h
M  +94   -1    server/src/collectionscheduler.cpp
M  +4    -1    server/src/collectionscheduler.h
M  +3    -0    server/src/handler/colcopy.cpp
M  +3    -0    server/src/handler/colmove.cpp
M  +3    -0    server/src/handler/copy.cpp
M  +3    -0    server/src/handler/fetch.cpp
M  +4    -0    server/src/handler/modify.cpp
M  +3    -0    server/src/handler/move.cpp

http://commits.kde.org/akonadi/ff9edf8c0d96b179116558020d5a571a25142795

-- 
You are receiving this mail because:
You are the assignee for the bug.