<html>
<body>
<div style="font-family: Verdana, Arial, Helvetica, Sans-Serif;">
<table bgcolor="#f9f3c9" width="100%" cellpadding="8" style="border: 1px #c9c399 solid;">
<tr>
<td>
This is an automatically generated e-mail. To reply, visit:
<a href="https://git.reviewboard.kde.org/r/117975/">https://git.reviewboard.kde.org/r/117975/</a>
</td>
</tr>
</table>
<br />
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<p style="margin-top: 0;">On May 29th, 2014, 12:33 p.m. UTC, <b>Commit Hook</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">This review has been submitted with commit ea0b761fbe3c43b42f83c5477018e10209f92ea3 by Martin Steigerwald to branch KDE/4.13.</pre>
</blockquote>
<p>On May 29th, 2014, 3:17 p.m. UTC, <b>Martin Steigerwald</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">My changes breaks two unittests:
16 - maildir-testmaildir (Failed)
30 - mixedmaildir-itemfetchtest (Failed)
I researched the maildir related one:
FAIL! : MaildirTest::testMaildirFlagsReading() Compared values are not the same
Actual (flags.count()): 0
Expected (1): 1
Loc: [/home/martin/KDE/Dev/kdepim-runtime/resources/maildir/libmaildir/tests/testmaildir.cpp(363)]
362 Akonadi::Item::Flags flags = d.readEntryFlags( entries[0] );
363 QCOMPARE( flags.count(), 1 );
364 QVERIFY( flags.contains( Akonadi::MessageFlags::Forwarded ) );
I don´t get why it fails. I thought due to keycache.cpp requiring sorting in methods listNew and listCurrent, but readEntryFlags doesn´t seem to call into keycache. Any hints?
I will revert patches at least from KDE/4.13 branch if I don´t find why its breaking and thus not able to fix it.
Heck, I think I got it:
The test uses: entries[0] which isn´t guarenteed to test the right entry in an unsorted list.</pre>
</blockquote>
<p>On May 29th, 2014, 3:52 p.m. UTC, <b>Martin Steigerwald</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Review request 118400 for libmaildir test case.</pre>
</blockquote>
</blockquote>
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Review request #118441 for mixedmaildir test case.</pre>
<br />
<p>- Martin</p>
<br />
<p>On May 20th, 2014, 10:40 p.m. UTC, Martin Steigerwald wrote:</p>
<table bgcolor="#fefadf" width="100%" cellspacing="0" cellpadding="8" style="background-image: url('https://git.reviewboard.kde.org/static/rb/images/review_request_box_top_bg.ab6f3b1072c9.png'); background-position: left top; background-repeat: repeat-x; border: 1px black solid;">
<tr>
<td>
<div>Review request for KDEPIM, KDEPIM-Libraries, Andras Mantia, David Faure, and Sergio Luis Martins.</div>
<div>By Martin Steigerwald.</div>
<p style="color: grey;"><i>Updated May 20, 2014, 10:40 p.m.</i></p>
<div style="margin-top: 1.5em;">
<b style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Bugs: </b>
<a href="http://bugs.kde.org/show_bug.cgi?id=334218">334218</a>
</div>
<div style="margin-top: 1.5em;">
<b style="color: #575012; font-size: 10pt;">Repository: </b>
kdepim-runtime
</div>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Description </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">On trying to find the cause for Akonadi maildir resource hogging up a Sandybridge core for minutes without any notable MySQL or disk activity, Sergio recommended doing a callgrind run. This revealed a bottleneck with QDir.entryList and there specifically in QAlgorihmsPrivate::qSortHelper[1].
Calling setSorting(QDir::NoSort) on the QDir object gets rid of the sorting. This I did twice in maildir.ccp and twice in keycache.cpp for listCurrent and listNew functions there.
It is not yet clear whether this change is safe, but on my tests it appears to do. Moving and filtering mails still work okay. I will CC Andras on this as Sergio tried to ping him about this change.
I think it is fair to say that this change has a *huge* impact on Akonadi maildir performance with large maildir and I recommend to backport this for stable branch as well once changing to no sorting is considered to be safe.
Thanks to Sergio and David for pointing out the sorting issue, telling how to change to no sorting and general help.
See also:
Bug 334218 - synchronizations of large folders with filesystem contents hogs a Sandybridge core for minutes
https://bugs.kde.org/334218
[1] https://bugs.kde.org/show_bug.cgi?id=334218#c4
[2] Bug 334206 - While maildir resources synchronizes a folder KMail blocks on switching to a different folder:
https://bugs.kde.org/334206</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Testing </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">In my tests with KMail SC 4.12.4 Debian unstable packages and Akonadi and kdepimlibs from Git it works well. Well basically Akonadi maildir resource changed from being a CPU hog for minutes to hardly every appearing in a 10 second measurement average snapshot of atop anymore. KMail hardly blocks on folder changes anymore, which I still believe to be a different issue tough[2].
Mail filtering and moving mails to a different folder still works ago. And KMail is subjectively much faster. It completely changes my KMail experience from being unbearable at times to quite pleasant.</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Diffs</b> </h1>
<ul style="margin-left: 3em; padding-left: 0;">
<li>resources/maildir/libmaildir/keycache.cpp <span style="color: grey">(f0af9c47cef63dac45ae68dd94fa17a040dc1593)</span></li>
<li>resources/maildir/libmaildir/maildir.cpp <span style="color: grey">(9bd380201a1a250ecfe354dd99946e8dae0ab668)</span></li>
</ul>
<p><a href="https://git.reviewboard.kde.org/r/117975/diff/" style="margin-left: 3em;">View Diff</a></p>
</td>
</tr>
</table>
</div>
</body>
</html>