[Akonadi] [Bug 334218] synchronizations of large folders with filesystem contents hogs a Sandybridge core for minutes

[Martin Steigerwald]

https://bugs.kde.org/show_bug.cgi?id=334218

--- Comment #18 from Martin Steigerwald <Martin at Lichtvoll.de> ---
Git commit afdc4bc8d85dbadfddd5d89169fede0f5a57317d by Martin Steigerwald.
Committed on 03/05/2014 at 14:52.
Pushed by steigerwald into branch 'KDE/4.13'.

Do not sort the directory entry list in listNew() and listCurrent():

According to callgrind dumps at

https://bugs.kde.org/show_bug.cgi?id=334218#c4

QDir sorts the list of directory entries unless specified otherwise.

After disabling the sorting the callgrind is quite different already:

https://bugs.kde.org/show_bug.cgi?id=334218#c7

This already helps shortening the time to synchronize folders visibly,
but KMail gets still blocked for half a minute or more. There is another
occurence of QDir entry sorting in keycache.cpp which the next commit
will address.

Thanks to Sergio and David for help and pointing out how to disable
the sorting.

REVIEW: 117975

DIGEST: Huge performance improvement for POP3 users with large maildir.

M  +2    -0    resources/maildir/libmaildir/maildir.cpp

http://commits.kde.org/kdepim-runtime/afdc4bc8d85dbadfddd5d89169fede0f5a57317d

--- Comment #19 from Martin Steigerwald <Martin at Lichtvoll.de> ---
Git commit ea0b761fbe3c43b42f83c5477018e10209f92ea3 by Martin Steigerwald.
Committed on 03/05/2014 at 14:53.
Pushed by steigerwald into branch 'KDE/4.13'.

Also do not sort directory entries in listCurrent and listNew in key cache:

QDir sorts the list of directory entries unless specified otherwise.
The last commit already disabled the sorting in maildir.cpp.

This commit completes the work and makes the remaining sorting calls
in the callgrind dumps at

https://bugs.kde.org/show_bug.cgi?id=334218#c7

go away completely as demonstrated in the callgrind dumps at:

https://bugs.kde.org/show_bug.cgi?id=334218#c12

Subjectively this has a huge impact on the performance of KMail with huge
maildir. KMail hardly blocks anymore on folder changes and feels much more
responsive now.

Akonadi maildir resource went from hogging a Sandy Bridge core for minutes
to not appearing using 100% of one core in even one averaged 10 second
interval in atop. I hardly see it in atop at all anymore, except when
accessing very large folders such as one with Linux Kernel mailing list
and more than 245000 unread mails.

Thanks to Sergio for pointing out the remaining sorting calls in the second
callgrind dump and for help.

Now the bottleneck appears to be MaildirResource::listRecursive but I do
not know whether further optimization is necessary at this point.

Both changes tested with kdepimlibs master as of today and KMail 4.12.4
from Debian unstable packages with an enormous maildir including a
folder for Linux Kernel mailing list with more than 245000 unread mails.

REVIEW: 117975

DIGEST: Huge performance improvement for POP3 users with large maildirs.

M  +2    -0    resources/maildir/libmaildir/keycache.cpp

http://commits.kde.org/kdepim-runtime/ea0b761fbe3c43b42f83c5477018e10209f92ea3

-- 
You are receiving this mail because:
You are the assignee for the bug.