[Akonadi] [Bug 319157] New: Excessive I/O usage during mail import with KMailCVT.
Martin Steigerwald
Martin at Lichtvoll.de
Wed May 1 11:24:01 BST 2013
https://bugs.kde.org/show_bug.cgi?id=319157
Bug ID: 319157
Summary: Excessive I/O usage during mail import with KMailCVT.
Classification: Unclassified
Product: Akonadi
Version: 4.10
Platform: Debian unstable
OS: Linux
Status: UNCONFIRMED
Severity: grave
Priority: NOR
Component: Maildir Resource
Assignee: kdepim-bugs at kde.org
Reporter: Martin at Lichtvoll.de
I am setting this to grave as this behaviour may destroy SSDs.
It appears to me that bug #315947 is not yet fixed. But my issue might be
something different as well, cause I am seeing this behaviour after having run
KMailCVT.
I am currently on my second try to migrate from KDEPIM 4.4.11 to KDEPIM 4.10.2
on my Debian Sid installation. This time I use the recommended approach to
start from scratch and import my mails:
martin at merkaba:~/Backup> du -sh Mail-Alt
6,7G Mail-Alt
martin at merkaba:~/Backup> find Mail-Alt -type f | wc -l
447779
martin at merkaba:~/Backup> find Mail-Alt -type d | wc -l
840
martin at merkaba:~/Backup>
atop reports my that the maildir agent already used up 368,7 GB.
This is 61x times the amount of space of the KMail-1 mixded maildir I imported.
I made sure that there are no mbox files in it.
Reproducible: Didn't try
Steps to Reproduce:
1. Do a KDEPIM-2 setup from scratch
2. Disable mail indexing
3. Don´t run the migrator
4. Use KMailCVT to import old mails (without deleting duplicates)
Actual Results:
KMailCVT completes pretty fast, but KMail-2 keeps unresponsive and I see this
since about an hour already:
martin at merkaba:~/Backup> vmstat 1
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
2 2 18496 1492104 88412 2737616 1 2 880 1551 113 54 17 2 79 1
2 0 18496 1442568 88412 2787688 0 0 80 69088 3805 16017 43 8 44
6
2 1 18496 1439100 88420 2790100 0 0 76 103680 3233 13271 36 7 46
10
2 0 18496 1475372 88420 2755028 0 0 132 69096 3257 14271 33 7 47
13
1 1 18496 1482848 88420 2746472 0 0 112 103788 3407 15593 36 7 46
12
1 1 18496 1468900 88420 2760732 0 0 88 105392 3510 14642 39 7 44
10
2 0 18496 1444772 88420 2784908 0 0 16 69112 3956 19002 41 11 38
10
3 0 18496 1430400 88428 2799604 0 0 76 69164 3439 14195 39 9 45
8
0 2 18496 1479756 88428 2750136 0 0 40 103680 3582 14124 32 8 45
15
4 0 18496 1477912 88428 2751936 0 0 356 69128 2980 12293 37 6 50
7
1 1 18496 1483032 88428 2745756 0 0 100 103700 3444 13683 34 10 44
12
3 1 18496 1498952 88428 2729880 0 0 60 69140 3278 12623 37 8 47
9
2 0 18496 1480140 88436 2749428 0 0 56 69192 3401 13724 39 7 46
8
0 2 18496 1493036 88436 2735732 0 0 80 103732 3264 14039 36 7 47
10
2 0 18496 1485352 88436 2744328 0 0 72 69160 3240 13224 36 7 49
9
1 1 18496 1486988 88436 2741568 0 0 108 103884 3575 16732 38 7 44
11
1 1 18496 1495584 88436 2733020 0 0 72 69168 3479 14323 38 7 47
8
2 0 18496 1479380 88444 2750100 0 0 84 69224 3363 14393 39 7 46
7
0 2 18496 1437836 88444 2790768 0 0 88 103776 3643 15913 38 11 41
11
martin at merkaba:/home2/martin#130> date; while true; do ls -lh
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat ;
sleep 1 ; done
Mi 1. Mai 12:19:55 CEST 2013
-rw-r--r-- 1 martin martin 13M Mai 1 12:19
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 26M Mai 1 12:19
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 30M Mai 1 12:19
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 1,6M Mai 1 12:19
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 25M Mai 1 12:19
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 0 Mai 1 12:20
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 12M Mai 1 12:20
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 23M Mai 1 12:20
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 8,9M Mai 1 12:20
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 35M Mai 1 12:20
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 0 Mai 1 12:20
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 22M Mai 1 12:20
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 32M Mai 1 12:20
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 29M Mai 1 12:20
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 0 Mai 1 12:20
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 8,8M Mai 1 12:20
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 30M Mai 1 12:20
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 0 Mai 1 12:20
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 25M Mai 1 12:20
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 0 Mai 1 12:20
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 17M Mai 1 12:20
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 32M Mai 1 12:20
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 0 Mai 1 12:20
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 15M Mai 1 12:20
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 0 Mai 1 12:20
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 5,7M Mai 1 12:20
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 13M Mai 1 12:20
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
-rw-r--r-- 1 martin martin 33M Mai 1 12:20
Akonadi/Konfiguration/agent_config_akonadi_maildir_resource_0_changes.dat
^C%
ATOP - merkaba 2013/05/01 12:20:38 ---------
1d20h57m16s elapsed
PRC | sys 66m30s | user 2h51m | #proc 273 | #tslpu 0 | #zombie
0 | #exit 0 |
CPU | sys 9% | user 70% | irq 0% | idle 314% | wait
6% | curscal ?% |
cpu | sys 2% | user 34% | irq 0% | idle 62% | cpu002 w
1% | curscal ?% |
cpu | sys 2% | user 22% | irq 0% | idle 75% | cpu003 w
1% | curscal ?% |
cpu | sys 2% | user 7% | irq 0% | idle 89% | cpu000 w
2% | curscal ?% |
cpu | sys 2% | user 7% | irq 0% | idle 89% | cpu001 w
2% | curscal ?% |
CPL | avg1 2.31 | avg5 2.38 | avg15 2.55 | csw 407966e3 | intr
11969e4 | numcpu 4 |
MEM | tot 7.6G | free 1.3G | cache 2.7G | buff 86.9M | slab
1.2G | shmem 250.7M |
SWP | tot 12.0G | free 12.0G | | | vmcom
3.7G | vmlim 15.8G |
PAG | scan 15480e3 | steal 1507e4 | stall 5092 | | swin
51794 | swout 145699 |
LVM | erkaba-home2 | busy 3% | read 44961 | write 1995e3 | MBw/s
2.87 | avio 1.04 ms |
LVM | merkaba-home | busy 3% | read 2709212 | write 425612 | MBw/s
0.19 | avio 0.65 ms |
LVM | rkaba-debian | busy 1% | read 666090 | write 670366 | MBw/s
0.17 | avio 0.33 ms |
LVM | merkaba-swap | busy 0% | read 52118 | write 145701 | MBw/s
0.00 | avio 0.06 ms |
DSK | sda | busy 5% | read 1951569 | write 1968e3 | MBw/s
3.23 | avio 0.90 ms |
NET | transport | tcpi 693664 | tcpo 497845 | udpi 68906 | udpo
41345 | tcpao 5246 |
NET | network | ipi 880745 | ipo 539020 | ipfrw 0 | deliv
879128 | icmpo 410 |
NET | eth0 0% | pcki 1076002 | pcko 493729 | si 47 Kbps | so 2
Kbps | erro 0 |
NET | lo ---- | pcki 47437 | pcko 47437 | si 3 Kbps | so 3
Kbps | erro 0 |
*** system and process activity since boot ***
PAUSED
PID TID THR SYSCPU USRCPU VGROW RGROW RDDSK WRDSK ST EXC S CPUNR
CPU CMD 1/12
566 - 8 1m53s 65m00s 653.0M 528.2M 7356K 1.1G N- - S 0
6% virtuoso-t
501 - 2 14m31s 45m58s 364.8M 145.7M 432K 405.3G N- - R 3
5% akonadi_agent_
Yes, thats 405,3GB for:
merkaba:~> ps aux | grep 501 | grep -v grep | grep -v atop
martin 501 63.2 1.8 373728 149232 ? Dl 10:44 60:57
/usr/bin/akonadi_agent_launcher akonadi_maildir_resource
akonadi_maildir_resource_0
merkaba:~> ps aux | grep 501 | grep -v grep | grep -v atop
martin 501 63.2 1.8 373728 149496 ? Dl 10:44 60:59
/usr/bin/akonadi_agent_launcher akonadi_maildir_resource
akonadi_maildir_resource_0
merkaba:~> ps aux | grep 501 | grep -v grep | grep -v atop
martin 501 63.2 1.8 373728 149496 ? Dl 10:44 61:00
/usr/bin/akonadi_agent_launcher akonadi_maildir_resource
akonadi_maildir_resource_0
merkaba:~> ps aux | grep 501 | grep -v grep | grep -v atop
martin 501 63.2 1.8 373728 149496 ? Dl 10:44 61:00
/usr/bin/akonadi_agent_launcher akonadi_maildir_resource
akonadi_maildir_resource_0
merkaba:~> ps aux | grep 501 | grep -v grep | grep -v atop
martin 501 63.2 1.8 373728 149496 ? Rl 10:44 61:00
/usr/bin/akonadi_agent_launcher akonadi_maildir_resource
akonadi_maildir_resource_0
Often in D state.
Akonadi has overwritten my Intel SSD 320 once already.
Expected Results:
A *sane* amount of I/O.
merkaba:~> cat /proc/version
Linux version 3.9.0-tp520 (martin at merkaba) (gcc version 4.7.2 (Debian 4.7.2-5)
) #7 SMP PREEMPT Mon Apr 29 15:01:05 CEST 2013
I use BTRFS as home partition, but placed ~/.config/akonadi,
~/.local/share/akonadi and ~/.local/share/local-mail on Ext4 partition.
--
You are receiving this mail because:
You are the assignee for the bug.
More information about the Kdepim-bugs
mailing list