[Kde-pim] Experiences with current Akonadi perf improvements

[Daniel Vrátil]

On Thursday, December 18, 2014 01:03:42 PM Martin Steigerwald wrote:
> Hello!
> 
> I have been testing c733429f4fa9696fb027ddc946e54f6bbb68deaf from 1.13
> branch of Akonadi and just as a short heads up:
> 
> It works nicely.
> 
> I see lot less load on MySQL.
> 
> By magnitudes.
> 
> So thats great!
> 
> Thank you.
> 
> 
> Yet it doesn´t solve some other existing issues with current Akonadi:
> 
> What I still see at a time is that Akonadi doesn´t respond to KMail in a
> timely manner, up to the point that KMail even looses the connection to
> Akonadi and then both sit there doing nothing anymore unless I restart
> KMail. I would like to easily see what Akonadi is doing at that time. Maybe
> you have some ideas on how I can see it.

KMail cannot really lose connection to Akonadi. Most likely the session get's 
blocked on a DBus call, which times out, leaving a dangling task in 
ResourceScheduler.


> I see akonadi maildir resource with high load then, sometimes MySQL at
> around 100-150%, but more often just maildir resource and akonadiserver. It
> may be synchronizing folders at that time, I sometimes don´t know exactly,
> cause often Akonadiconsole doesn´t provide any useful feedback on what the
> resource is doing at the moment. Sometimes I see its synchronizing a
> folder, sometimes I see no info in Akonadiconsole while it still hogs a CPU
> core for a while.

The problem with maildir is that we don't hold it stored in Akonadi for very 
long - the data are loaded on demand from maildir into Akonadi and dropped 
soon after to reduce disk usage (the file is already once locally in the 
maildir, no reason to keep it duplicated in Akonadi). This works reasonably 
for reasonably-sized maildirs, but of course fails utterly on massive 
maildirs. I don't see how this could be solved easily, other than disabling 
the cache expiration for maildir folders.


> I remember that I optimized the maildir scan by stopping it from sorting the
> maildir when trying to list it. Yet I still wonder why on earth it
> synchronizes at all:  And in what circumstances. If Akonadi stored the
> files there, they will be there tomorrow as well – thats the whole purpose
> a filesystem is about. So during runtime of Akonadi it has inotify on them
> should another app add a mail there, so there is no business to *ever*
> synchronize a maildir at all during runtime of Akonadi, except on the first
> time a folder is accessed after startup of Akonadi where it may check
> whether an app dropped a mail there while Akonadi was stopped.
> 
> I hope in the next time I have some more time to actually at least try to
> diagnose the KMail Akonadi connection breakup. Its the most annoying thing
> for me currently as it happens one or two times a day at least usually.
> 
> So thats important points for Akonadi Next IMHO:
> 
> 1) Provide good progress report on what each resource is doing. A user who
> sees a process hogging a CPU core without *any* feedback as to *why* will
> likely get dissatisfied with it. Of course, if that kind of thing doesn´t
> happen with Akonadi Next anymore, this point could be skipped. Or progress
> report made less often in order to not harm the performance by progress
> reporting.

Akonadi does not see into resources, it's up to resources to report what they 
are doing. And often "less is more" in this case. It's better to just say 
"Synchronizing emails" instead of having tons of random status messages that 
don't really say anything useful. Having descriptive status messages is not a 
solution for the problem at hand. The solution is to fix the resources not to 
hog cpu at 100% for hours.


> 2) Do only work that is *necessary*. Be as lazy as you can get. As I
> suspected for a long time and as recent optimization work has shown:
> Current Akonadi did a lot of work that wasn´t necessary. And I think in
> case of folder synchronization it still does useless work.
> 
> Ciao,

-- 
Daniel Vrátil | dvratil at redhat.com | dvratil on #kde-devel, #kontact, #akonadi
Software Engineer - KDE Desktop Team, Red Hat Inc.

GPG Key: 0xC59D614F6F4AE348
Fingerprint: 4EC1 86E3 C54E 0B39 5FDD B5FB C59D 614F 6F4A E348
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-pim/attachments/20141218/138d578b/attachment.sig>
-------------- next part --------------
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/