[Bug 294772] New: Akonadi IMAP/maildir resource agent writes extensively for long period after deleting a several thousand e-mails

[Christian Burger]

https://bugs.kde.org/show_bug.cgi?id=294772

           Summary: Akonadi IMAP/maildir resource agent writes extensively
                    for long period after deleting a several thousand
                    e-mails
           Product: kmail2
           Version: 4.7
          Platform: Gentoo Packages
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: NOR
         Component: general
        AssignedTo: kdepim-bugs at kde.org
        ReportedBy: christian.burger at gmail.com


Created an attachment (id=69073)
 --> (http://bugs.kde.org/attachment.cgi?id=69073)
decompressed about 2 megabytes of strace log

Version:           4.7 (using KDE 4.7.4) 
OS:                Linux

I am using Akonadi 1.6.2 and KMail 4.7.4 on a Gentoo platform. I deleted
several thousand files in an IMAP folder (with activated disconnected feature)
and another thousand files in my local trash folder. This resulted in heavy
disk I/O about 15-20 MBytes/sec for at least 15 minutes, probably longer.

$ dstat 
----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw
 22  10  41  26   0   0| 443k 6371k|   0     0 |   0     0 | 598   704
 38  46   0  16   0   1|   0    37M|   0     0 |   0     0 |1718   731
 18  17   0  64   0   1|   0    14M|   0     0 |   0     0 | 739   659
 28  30   0  41   0   1|   0    26M|   0     0 |   0     0 |1319   545
 19  22   0  58   0   0|   0    18M|   0     0 |   0     0 | 919   407
 23  24   0  52   0   1|   0    17M|   0     0 |   0     0 | 889   614
 23  23   0  53   0   1|   0    19M|   0     0 |   0     0 | 868   604
 29  23   0  48   0   0|   0    19M|   0     0 |   0     0 | 980   501
  9   7   0  83   0   1|   0  4880k|   0     0 |   0     0 | 259   171
 11   8   0  81   0   0|   0  5140k|   0     0 |   0     0 | 313   587
 17  12   0  71   0   0|   0    10M|   0     0 |   0     0 | 521   337
  6   5   0  89   0   0|   0  2262k|   0     0 |   0     0 | 222   270
 12   8   0  80   0   0|   0  5392k|   0     0 |   0     0 | 374   598

I traced the culprits to the Akonadi processes associated with the mail folders
from where I deleted the mails: 

Both processes were truncating (down to zero) and writing one file each over
and over again:

1.
process as shown in ps: "/usr/bin/akonadi_agent_launcher
akonadi_maildir_resource akonadi_maildir_resource_2"
path to heavy i/o file:
~/.config/akonadi/agent_config_akonadi_maildir_resource_2_changes.dat

2.
ps: "/usr/bin/akonadi_imap_resource --identifier akonadi_imap_resource_12"
path: ~/.config/akonadi/agent_config_akonadi_imap_resource_12_changes.dat 

At the beginning (at least the moment when I realized something was wrong) the
latter file was about 200 KBytes and the former about 2 MBytes. The file size
was shrinking each truncate/write cycle by a small amount (around 1
kilobytes/sec). I see no need for I/O of that magnitude, just because I deleted
a huge bunch of obsolete e-mails.

Just tried to reproduce the problem and it seems that pretty much any huge
change to a mail folder will have this effect. I copied an IMAP folder of a
thousand e-mails to my local mail folder and the fun begun.

Using a SSD I am pretty sure that this behaviour will shorten the lifespan of
my disk unneccessarily. By now I have migrated the problematic files
temporarily to a classic hard disk and will stay away from huge changes in my
mail folders without a hard disk at hand.

PS: I thought Linux's disk cache would take care of situations like that, but
my HDD LED is flickering like crazy.

Reproducible: Always

Steps to Reproduce:
either
delete a huge number (more than 1000) of e-mails from one folder
or
copy a huge number of e-mails from a IMAP folder to a local mail folder

Actual Results:  
seemingly unnecessary I/O

Expected Results:  
much less I/O

I have attached an unfiltered strace from one of the processes going wild.

Maybe bug #293175 is related to mine, but the information given there is too
little to be sure.

-- 
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.