[Akonadi] [Bug 321944] New: synchronizing large about 77000 mail Linux kernel mailing list folder blocks out KMail for more than 15 minutes
Martin Steigerwald
Martin at Lichtvoll.de
Thu Jul 4 10:31:28 BST 2013
https://bugs.kde.org/show_bug.cgi?id=321944
Bug ID: 321944
Summary: synchronizing large about 77000 mail Linux kernel
mailing list folder blocks out KMail for more than 15
minutes
Classification: Unclassified
Product: Akonadi
Version: 1.9.2
Platform: Debian unstable
OS: Linux
Status: UNCONFIRMED
Severity: normal
Priority: NOR
Component: Maildir Resource
Assignee: kdepim-bugs at kde.org
Reporter: Martin at Lichtvoll.de
Due to bugs 319226 and 320041 I still filter my mails manually. I just let them
flow into a special inbox folder, weed out spam manually and then press Ctrl-A
and Ctrl-J to manually trigger filtering.
This worked quite well. But now Linux/kernel-ml folder has about 77000 mails
and synchronizing it takes *ages*. During this synchronization KMail is almost
unusable. It takes minutes for a mail to display or delete.
Reproducible: Always
Steps to Reproduce:
1. Have a high performance system with Sandybridge i5 2520-M at 2,5 GHz and
Intel SSD 320 or similar
2. Have a maildir setup with a kernel mailing list folder of about 77000 mails
3. Have it download and filter mail after a day of non usage.
Actual Results:
merkaba:~> atopsar -O -b 0:00 -e 1:00
merkaba 3.10.0-tp520 #18 SMP PREEMPT Tue Jul 2 09:41:49 CEST 2013 x86_64
2013/07/04
-------------------------- analysis date: 2013/07/04 --------------------------
00:00:02 pid command cpu% | pid command cpu% | pid command cpu%_top3_
00:10:02 7246 akonadi_ 4% | 7191 mysqld 4% | 15622 kmail 2%
00:20:02 15622 kmail 2% | 1929 Xorg 2% | 2464 kwin 1%
00:30:02 7234 akonadi_ 9% | 7191 mysqld 9% | 15622 kmail 3%
00:36:21 7234 akonadi_ 94% | 7191 mysqld 29% | 7189 akonadis 19%
hibernation cycle
merkaba:~> atopsar -O -b 10:30 -e 11:30
merkaba 3.10.0-tp520 #18 SMP PREEMPT Tue Jul 2 09:41:49 CEST 2013 x86_64
2013/07/04
-------------------------- analysis date: 2013/07/04 --------------------------
10:51:12 pid command cpu% | pid command cpu% | pid command cpu%_top3_
11:01:12 7234 akonadi_ 94% | 7191 mysqld 25% | 7189 akonadis 16%
11:11:12 7234 akonadi_ 95% | 7191 mysqld 34% | 7189 akonadis 16%
atopsar measures *averages*. So that means that on 11:11:12 akonadi maildir
resource agent used up 94% on average for 10 minutes! And before that for
further 10 minutes and before that before hibernating for 6 minutes.
So 26 minutes of using one core for synchronizing 77000 mails.
26 minutes while KMail is almost completely unusable. Filtering almost hangs,
accessing mail takes minutes and so on.
Expected Results:
1. KMail remains snappy whatever Akonadi is doing in the background.
2. The CPU usage should match the task at hand. If our Zimbra server at work
took 26 minutes for synchronizing a mail folder of 77000 mails it wouldn´t to
much else. I have a kernel-ml folder there of above >330000 mails. And there
are upto 45 other users on the *same* server, not just me. But anyway, it
follows point one closely, so I do not even care to much at how much time it is
spending on background tasks. But when I see that on accessing the folder the
contents appear recent I think it is *way faster*.
I only set local mail folders maildir resource to synchronize at start of KMail
only and not at every POP3 download as well (cause thats unusable slow then).
So as I have stopped and started Akonadi yesterday in order to save some memory
for playing PlaneShift, it may have been that initial maildir sync. I think I
will disable this one as well as a work-around. Since only Akonadi accesses
those maildir folders I do not get why it needs to synchronize any folders at
all. If Akonadi moves a mail I expect it to know what it has done.
I optimized MySQL a bit:
# memory buffer InnoDB uses to cache data and indexes of its tables
(default:128M)
# Larger values means less I/O
# HINT: Raised from 80 MiB to a huge 500 MiB to see whether it makes a
difference, 2.5.2013
# HINT: Lowered from 500 MiB to 200 MiB, the original value to try I thought
about, 3.5.2013
innodb_buffer_pool_size=200M
It has been at a ridicolously low 64 MB. But since its not MySQL or I/O load
spiking out here, I think the current value works quite okay.
Database size is:
martin at merkaba:~/.local/share/akonadi/db_data/akonadi> ls -lh
insgesamt 1,2G
-rw-rw---- 1 martin martin 8,5K Mai 1 16:07 collectionattributetable.frm
-rw-rw---- 1 martin martin 128K Mai 20 13:07 collectionattributetable.ibd
-rw-rw---- 1 martin martin 8,5K Mai 1 16:07 collectionmimetyperelation.frm
-rw-rw---- 1 martin martin 144K Jul 1 22:47 collectionmimetyperelation.ibd
-rw-rw---- 1 martin martin 8,5K Mai 1 16:07 collectionpimitemrelation.frm
-rw-rw---- 1 martin martin 112K Mai 1 16:07 collectionpimitemrelation.ibd
-rw-rw---- 1 martin martin 42K Mai 1 16:07 collectiontable.frm
-rw-rw---- 1 martin martin 160K Jul 4 11:29 collectiontable.ibd
-rw-rw---- 1 martin martin 61 Mai 1 16:07 db.opt
-rw-rw---- 1 martin martin 8,4K Mai 1 16:07 flagtable.frm
-rw-rw---- 1 martin martin 112K Mai 20 11:55 flagtable.ibd
-rw-rw---- 1 martin martin 8,4K Mai 1 16:07 mimetypetable.frm
-rw-rw---- 1 martin martin 112K Mai 1 16:07 mimetypetable.ibd
-rw-rw---- 1 martin martin 8,6K Mai 1 16:07 parttable.frm
-rw-rw---- 1 martin martin 1,1G Jul 4 11:29 parttable.ibd
-rw-rw---- 1 martin martin 8,5K Mai 1 16:07 pimitemflagrelation.frm
-rw-rw---- 1 martin martin 20M Jul 4 11:10 pimitemflagrelation.ibd
-rw-rw---- 1 martin martin 8,7K Mai 1 16:07 pimitemtable.frm
-rw-rw---- 1 martin martin 92M Jul 4 11:29 pimitemtable.ibd
-rw-rw---- 1 martin martin 8,5K Mai 1 16:07 resourcetable.frm
-rw-rw---- 1 martin martin 112K Mai 20 11:27 resourcetable.ibd
-rw-rw---- 1 martin martin 8,4K Mai 1 16:07 schemaversiontable.frm
-rw-rw---- 1 martin martin 96K Mai 1 16:07 schemaversiontable.ibd
martin at merkaba:~/.local/share/akonadi/db_data> ls -lh ib*
-rw-rw---- 1 martin martin 114M Jul 4 11:29 ibdata1
-rw-rw---- 1 martin martin 64M Jul 4 11:29 ib_logfile0
-rw-rw---- 1 martin martin 64M Jul 3 15:03 ib_logfile1
--
You are receiving this mail because:
You are the assignee for the bug.
More information about the Kdepim-bugs
mailing list