<html>
<body>
<div style="font-family: Verdana, Arial, Helvetica, Sans-Serif;">
<table bgcolor="#f9f3c9" width="100%" cellpadding="8" style="border: 1px #c9c399 solid;">
<tr>
<td>
This is an automatically generated e-mail. To reply, visit:
<a href="https://git.reviewboard.kde.org/r/117975/">https://git.reviewboard.kde.org/r/117975/</a>
</td>
</tr>
</table>
<br />
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<p style="margin-top: 0;">On May 5th, 2014, 3:08 p.m. UTC, <b>Sergio Luis Martins</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Commit to stable too ?</pre>
</blockquote>
<p>On May 5th, 2014, 3:32 p.m. UTC, <b>Laurent Montel</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Please wait after 4.13.1 it will tag the 8 may.
Apply it in 4.13.2 if safe
thanks</pre>
</blockquote>
<p>On May 5th, 2014, 9:16 p.m. UTC, <b>Martin Steigerwald</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Okay, will wait. I think its good if someone else than me actually also tests it. For me it works really nicely.</pre>
</blockquote>
</blockquote>
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Hi! So did anyone test with it? It works very well for me. I see higher I/O throughput on accessing folders, but I think thats just due to the fact that there is way less CPU overhead now. It will stat() mail files in random order instead of alphabetically now, I think. Depending on the filesystem implementation and the order in which those files will be created, this may make a difference – accessing in inode number order may help on some filesystems. In any case, subjectively I think it is way better than before. What do you experience?
So did anyone test and okay with committing to 4.13.2? How can I do it? Rebase my nosorting branch to 4.13.2 branch and then merge my nosorting branch with the two commits into 4.13.2 branch? I did so with my commits to master branch, so I think this would work.</pre>
<br />
<p>- Martin</p>
<br />
<p>On May 5th, 2014, 12:07 p.m. UTC, Martin Steigerwald wrote:</p>
<table bgcolor="#fefadf" width="100%" cellspacing="0" cellpadding="8" style="background-image: url('https://git.reviewboard.kde.org/static/rb/images/review_request_box_top_bg.ab6f3b1072c9.png'); background-position: left top; background-repeat: repeat-x; border: 1px black solid;">
<tr>
<td>
<div>Review request for KDEPIM, KDEPIM-Libraries, Andras Mantia, David Faure, and Sergio Luis Martins.</div>
<div>By Martin Steigerwald.</div>
<p style="color: grey;"><i>Updated May 5, 2014, 12:07 p.m.</i></p>
<div style="margin-top: 1.5em;">
<b style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Bugs: </b>
<a href="http://bugs.kde.org/show_bug.cgi?id=334218">334218</a>
</div>
<div style="margin-top: 1.5em;">
<b style="color: #575012; font-size: 10pt;">Repository: </b>
kdepim-runtime
</div>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Description </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">On trying to find the cause for Akonadi maildir resource hogging up a Sandybridge core for minutes without any notable MySQL or disk activity, Sergio recommended doing a callgrind run. This revealed a bottleneck with QDir.entryList and there specifically in QAlgorihmsPrivate::qSortHelper[1].
Calling setSorting(QDir::NoSort) on the QDir object gets rid of the sorting. This I did twice in maildir.ccp and twice in keycache.cpp for listCurrent and listNew functions there.
It is not yet clear whether this change is safe, but on my tests it appears to do. Moving and filtering mails still work okay. I will CC Andras on this as Sergio tried to ping him about this change.
I think it is fair to say that this change has a *huge* impact on Akonadi maildir performance with large maildir and I recommend to backport this for stable branch as well once changing to no sorting is considered to be safe.
Thanks to Sergio and David for pointing out the sorting issue, telling how to change to no sorting and general help.
See also:
Bug 334218 - synchronizations of large folders with filesystem contents hogs a Sandybridge core for minutes
https://bugs.kde.org/334218
[1] https://bugs.kde.org/show_bug.cgi?id=334218#c4
[2] Bug 334206 - While maildir resources synchronizes a folder KMail blocks on switching to a different folder:
https://bugs.kde.org/334206</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Testing </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">In my tests with KMail SC 4.12.4 Debian unstable packages and Akonadi and kdepimlibs from Git it works well. Well basically Akonadi maildir resource changed from being a CPU hog for minutes to hardly every appearing in a 10 second measurement average snapshot of atop anymore. KMail hardly blocks on folder changes anymore, which I still believe to be a different issue tough[2].
Mail filtering and moving mails to a different folder still works ago. And KMail is subjectively much faster. It completely changes my KMail experience from being unbearable at times to quite pleasant.</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Diffs</b> </h1>
<ul style="margin-left: 3em; padding-left: 0;">
<li>resources/maildir/libmaildir/keycache.cpp <span style="color: grey">(f0af9c47cef63dac45ae68dd94fa17a040dc1593)</span></li>
<li>resources/maildir/libmaildir/maildir.cpp <span style="color: grey">(9bd380201a1a250ecfe354dd99946e8dae0ab668)</span></li>
</ul>
<p><a href="https://git.reviewboard.kde.org/r/117975/diff/" style="margin-left: 3em;">View Diff</a></p>
</td>
</tr>
</table>
</div>
</body>
</html>