[Akonadi] [Bug 319208] New: KMail unresponsive for minutes after POP3 retrieval and filtering with high MySQL load for minutes

[Martin Steigerwald]

https://bugs.kde.org/show_bug.cgi?id=319208

            Bug ID: 319208
           Summary: KMail unresponsive for minutes after POP3 retrieval
                    and filtering with high MySQL load for minutes
    Classification: Unclassified
           Product: Akonadi
           Version: 4.10
          Platform: Other
                OS: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: NOR
         Component: general
          Assignee: kdepim-bugs at kde.org
          Reporter: Martin at Lichtvoll.de

I set component to general as I do not know which compoment fits.

After my fifth migration attempt seemed to be successful, I still have
performance issues. For this migration attempt, I put my old KMail-1 maildir
contents (no mbox files in there) into ~/.local/share/local-mail and started
with a fresh KMail configuration and newly created filter rules (no import).
This works so far without any data losses as I experienced in my attempts with
mutiple maildir resources (see bug #318444 and bug #318227, bug #318290, all on
BTRFS,  bug #319175 on Ext4). All Akonadi and Nepomuk data is stored on Ext 4
except for contacts and std.ics calender.

The performance issue is as follows: KMail becomes unresponsive for minutes
after a POP3 retrieval and mail filtering actions. I see mysql at 100-200% for
minutes. I have been a bit late at pointing mytop at it, but I intend to
collect some more data.

This happens on a ThinkPad T520 with a dualcore Intel i5-2520M with 2,50 GHz,
that the ThinkPad easily overclocks to up to 3,2 GHz for minutes, 8 GiB of RAM,
and a 300 GiB Intel SSD 320. This is a hardware were I consider such
responsiveness issues in a mail program as bug.

Reproducible: Always

Steps to Reproduce:
1. Create a pop account.
2. Stuff some 5 GB messages in 100 folders and some about 30 filter rules at it
3. Have quite some mailinglists flow into it, including high volume ones like
LKML, debian-user and such.
4. Wait a night.
5. Press button to receive mails.
Actual Results:  
KMail becomes unresponsive for minutes. It took at least a minute to actually
filter all mails out of the inbox, but then also remains unresponsive for at
least five more minutes.

Clicking on a mail yields nothing or just the blue wait folder. During that
MySQL uses up 100-200% of CPU time consistently:

merkaba:~> atopsar -O -b 10:20 -e 12:00 

merkaba  3.9.0-tp520  #7 SMP PREEMPT Mon Apr 29 15:01:05 CEST 2013  x86_64 
2013/05/02

-------------------------- analysis date: 2013/05/02 --------------------------

10:23:30    pid command  cpu% |   pid command  cpu% |   pid command  cpu%_top3_
10:33:30   2459 mysqld   110% |  2500 akonadi_  17% |  2758 kmail      8%
10:43:30   2459 mysqld   116% |  2500 akonadi_  23% | 32672 icewease   6%
10:53:30   2459 mysqld     8% |  2500 akonadi_   6% |  1851 Xorg       5%


MySQL was also a heavy disk user during that time:

merkaba  3.9.0-tp520  #7 SMP PREEMPT Mon Apr 29 15:01:05 CEST 2013  x86_64 
2013/05/02

-------------------------- analysis date: 2013/05/02 --------------------------

10:23:30    pid command  dsk% |   pid command  dsk% |   pid command  dsk%_top3_
10:33:30   2459 mysqld    59% | 32428 apt        5% |  5874 akregato   4%
10:43:30   2459 mysqld    32% | 32672 icewease  30% |  2500 akonadi_  12%
10:53:30      1 systemd   88% |  2459 mysqld     6% |   352 btrfs-tr   1%



atop created averages in its 10 minutes recordings.

I will include the atop data file again which is suitable for step by step
going through this again to look at whats happening.

Expected Results:  
- KMail remains responsive during POP3 retrieval and filtering
- The MySQL CPU usage and I/O usage remains within bounds that make sense for
this kind of workload

I know, what makes sense needs to be evaluated. I admit, I have a huge setup,
but I do think that the current resource usage for it is way over top.

What I will try:

1) Just selecting POP3 retrieval for this one main POP3 account in the menu.
Cause if I do press the icon in the toolbar, KMail also synchronized that huge
sized local maildir resource again, which IMHO doesn´t make any sense to do in
that case.

2) Raising some innodb buffer size limit from the IMHO very small 80 MiB to
huge 500 MiB to see whether it makes a difference. If it does, I will lower it
gradually until maybe about 200 MiB.

martin at merkaba:~/.local/share/akonadi> diff -u mysql.conf.orig-2013-05-02
mysql.conf
--- mysql.conf.orig-2013-05-02  2013-05-01 16:06:58.180288216 +0200
+++ mysql.conf  2013-05-02 11:01:45.024564563 +0200
@@ -42,7 +42,8 @@

 # memory buffer InnoDB uses to cache data and indexes of its tables
(default:128M)
 # Larger values means less I/O
-innodb_buffer_pool_size=80M
+# HINT: Raised from 80 MiB to a huge 500 MiB to see whether it makes a
difference, 2.5.2013
+innodb_buffer_pool_size=500M

 # Create a .ibd file for each table (default:0)
 innodb_file_per_table=1


I will try pressing the icon to have both happening at same time and just
triggering POP3 retrievel of the main account with the 500 MiB setting. It will
take some time, till I have more data.

-- 
You are receiving this mail because:
You are the assignee for the bug.