[Kde-pim] PIM is not good. Would adding an extra RC help?

Sat Apr 5 17:15:57 UTC 2014

For my part, I'm running whichever version is in openSUSE KDE:Unstable:SC and IMAP performance is slow.  My biggest issue is client side filtering.  Most of my filters won't run automatically for some reason, and if I select a group of 100 or so emails and apply filters manually, its taking kmail2 about ten minutes to slog through them.  My biggest issue is client-side spam filtering though.  Whether I use spamassassin or spambayes, they start deleting the bodies of my emails when filtering, both client and server side.  Which means I'm running without spam filtering, which is annoying, to say the least.

On Apr 5, 2014 9:58 AM, Martin Steigerwald <Martin at lichtvoll.de> wrote:
>
> Hi Albert,
>
> I am mostly a user of KDEPIM, but since I experienced quite some performance 
> issues and did some analysis and reports every now and then, I thought I share 
> my findings.
>
> Am Samstag, 5. April 2014, 16:40:24 schrieb Albert Astals Cid:
> > ** Please CC me and r-t list, i'm not subscribed to pim list. **
> > 
> > Hello guys, as I hope you all know this Wednesday we are supposed to tag the
> > final release for 4.13.
> > 
> > I am sending this email to ask for opinions on the value of adding an extra
> > RC and delaying the 4.13.0 release for at least a week.
> > 
> > I'm going to explain why I'd like this to happen.
> > 
> > Since the update to 4.13 Beta I've been having lots of CPU and I/O problems
> > in pim related stuff, most notably akonadi_baloo_indexer,
> > akonadi_maildir_resource & friends.
> > 
> > On every release i've complained on IRC to be told "oh yes, we just fixed
> > this thing and will be available on the next release".
> > 
> > But here I am, using 4.13 RC and with even KMail closed
> > mysqld+akonadiserver+akonadi_maildir_resource are using 100% of one of my
> > CPU cores and writing/reading around 10MB/s to my slow disks.
>
> So you are using a POP3 setup? Or how is the maildir resource filled on your 
> setup? How big is it?
>
> I reported several bugs regarding slowness of maildir resource with lots of 
> folders with lots of mails in it, including a report of 
> akonadi_maildir_resource hogging up one Intel Sandybridge core on a Dual SSD 
> BTRFS RAID 1 setup for several minutes after filtering about 100 mails into 
> their folders[1]. And one for MySQL tuning, but at least from my observation 
> the issue isn´t MySQL here, I thought so first, but I am not convinced after a 
> deeper look at show innodb engine in MySQL is doing. See recent thread in pim-
> ml as well[3]. I am keen to look at it some more, but after spending quite a 
> lot of time on analysing I needed a break. Well some other things Mark 
> found[4]
>
> But on the other hand my setup might be considered extreme. About 100 folders, 
> quite some of them with several ten thousand mails and one (kernel-ml) with 
> about 220000 unread mails.
>
> It is my impression that both for my private POP3 setup and my work IMAP setup 
> performance got way better on the upgrade from Akonadi 1.11.0 to Akonadi 
> 1.12.0. Still I see the akonadi_maildir_resource hogging a core for minutes at 
> times. But it doesn´t seem to happen after every mail retrieval / mail 
> filtering cycle. Sometimes MySQL chimes in using something between one and 
> three cores, but I see akonadi_maildir_resource for minutes on 100% CPU 
> without any notable CPU activity of mysqld as well.
>
> But when it does, I see it synchronizing folders. In strace I see it getting 
> dentries and checking the existance of every little file.
>
> I don´t know whether another week will lead to a fix. I got the impression that 
> performance issues with Akonadi are not yet fully understood.
>
> In my eyes there are two things which contribute a lot to KMail being 
> basically unusable for times of several minutes:
>
> 1) Maildir resource synchronizes folders needlessly. Not sure what triggers it 
> tough.
>
> 2) Akonadi blocks out KMail almost completely when maildir resource is doing 
> so. To me Akonadi appears to be basically single threaded. While I understand 
> one of its design goals of Akonadi were that GUI applications like KMail never 
> block again.
>
> Also remember for me this is Akonadi 1.12.0 and KDEPIM 4.12.4 still. But for 
> these versions, although I see improvements, I still think KDEPIM + Akonadi 
> needs serious performance related work. Cause I just don´t get how filtering 
> 100-200 mails to their folders can lead to a usage of one Sandybridge core for 
> minutes unless I assume somewhere something does unnecessary work. Its not a 
> I/O bottleneck according to atop.
>
> That said:
>
> - I don´t think the performance issues are new. So they are no regression. Or 
> did you found it regressing recently?
>
> - I am not sure whether another week would help with fixing them, as… my bet 
> is: If the cause is understood someone likely would have fixed them already.
>
> I do think the KDEPIM / Akonadi team needs helping hands. I tried to help a 
> bit, but I didn´t come around at looking at the code and when I do I first need 
> to understand some of it before I can consider proposing a patch. :)
>
> [1] [Akonadi] [Bug 332684] New: [Maildir] lots of stats calls to 
> /etc/localtime on synchronizing folders
>
> => can be work-arounded by setting TZ env var, yet folder synchronization 
> still is slow.
>
> [2] [Akonadi] [Bug 332626] MySQL tuning: adaption of MySQL tuning options for 
> larger accounts
>
> [3] [Kde-pim] Akonadi MySQL backend: tuning for larger accounts or switching 
> to MariaDB with a different storage engine?
>
> [4] PIM Sprint, Calendar progress and Akonadi
> Datum:Donnerstag 17:13 
> Autor:Mark Gaiser (markg)
>
> http://kdeblog.mageprojects.com/2014/04/03/pim-sprint-calendar-progress-and-akonadi/
>
> [5] Re: [Kde-pim] Akonadi MySQL backend: tuning for larger accounts or 
> switching to MariaDB with a…
>
> This is not the beginning of the thread, but lists.kde.org is very slow at the 
> moment.
>
> http://lists.kde.org/?l=kde-pim&m=139603704423385&w=2
>
> [Kde-pim] Slowness in POP3 case, maildir resource stat()ing on /etc/localtime 
> very often…
> http://lists.kde.org/?l=kde-pim&m=139592803810131&w=2
>
> [6] [Kde-pim] Also fetches all items on Maildir filtering? (was: Review Request 
> 117140: ItemSync: Only…
> http://lists.kde.org/?l=kde-pim&m=139629170212642&w=2
>
> Ciao,
> -- 
> Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
> GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7
> _______________________________________________
> KDE PIM mailing list kde-pim at kde.org
> https://mail.kde.org/mailman/listinfo/kde-pim
> KDE PIM home page at http://pim.kde.org/