[Akonadi] [Bug 364114] New: moving a folder within one maildir resource is extremely slow and inefficient

[Martin Steigerwald via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=364114

            Bug ID: 364114
           Summary: moving a folder within one maildir resource is
                    extremely slow and inefficient
           Product: Akonadi
           Version: 16.04
          Platform: Other
                OS: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: NOR
         Component: Maildir Resource
          Assignee: kdepim-bugs at kde.org
          Reporter: Martin at Lichtvoll.de

Today I dared moving .Computer.directory/mesa-dev-ml with about 86730 mails to
.Linux.directory by drag and drop within KMail. After the move operation
completed kmail crashed.

The operation took at least one and a half hour on a ThinkPad T520 with Intel
Sandybridge i5-2520m dual core CPU and dual SSD BTRFS RAID 1 and 16 GiB of RAM!

Reproducible: Always

Steps to Reproduce:
1. Have a large maildir folder inside another folder.
2. Move it to a different folder


Actual Results:  
1. Akonadi first moves all mails into database or file_db_data depending on
threshold. This process is extremely slow at a rate of about 20000-40000 mails
in half an hour. During that the mysqld database process frequently uses about
90-99% of one logical core. This process takes about 1,5 hours for 86000 mails.

martin at merkaba:~/.local/share/akonadi/file_db_data> date ; find -type f | wc -l
Mi 8. Jun 20:41:59 CEST 2016
30339
martin at merkaba:~/.local/share/akonadi/file_db_data> date ; find -type f | wc -l
Mi 8. Jun 20:42:07 CEST 2016
30473
martin at merkaba:~/.local/share/akonadi/file_db_data> date ; find -type f | wc -l
Mi 8. Jun 20:42:16 CEST 2016
30494
martin at merkaba:~/.local/share/akonadi/file_db_data> date ; find -type f | wc -l
Mi 8. Jun 20:42:32 CEST 2016
30708
martin at merkaba:~/.local/share/akonadi/file_db_data> date ; find -type f | wc -l
Mi 8. Jun 20:42:53 CEST 2016
31131
martin at merkaba:~/.local/share/akonadi/file_db_data> date ; find -type f | wc -l
Mi 8. Jun 20:45:58 CEST 2016
34562
martin at merkaba:~/.local/share/akonadi/file_db_data> date ; find -type f | wc -l
Mi 8. Jun 21:28:28 CEST 2016
67480

martin at merkaba:~/.local/share/akonadi/file_db_data> cd ..
martin at merkaba:~/.local/share/akonadi> date ; find file_db_data -type f | wc -l
; du -sh db_data 
Mi 8. Jun 21:28:58 CEST 2016
68099
4,9G    db_data
martin at merkaba:~/.local/share/akonadi> date ; find file_db_data -type f | wc -l
; du -sh db_data
Mi 8. Jun 21:33:16 CEST 2016
74039
5,0G    db_data
martin at merkaba:~/.local/share/akonadi> date ; find file_db_data -type f | wc -l
; du -sh db_data
Mi 8. Jun 21:45:13 CEST 2016
95529
5,1G    db_data


2. Then after moving completed it copies the mails to the destination folder.
This is quite quick. The mains still remain in file_db_data:

martin at merkaba:~/.local/share/akonadi> date ; find file_db_data -type f | wc -l
; du -sh db_data
Mi 8. Jun 21:53:26 CEST 2016
98737
5,1G    db_data

3. KMail crashes.

During the whole operation KMail does not respond to user input. It is
completely blocked.

Expected Results:  
When moving a folder within a local maildir resource, Akonadi does the
following:

1. Rename the folder.

2. Store the folder renaming within the database.

3. KMail remains responsive at all times.

In other words: The operation is completed within 10 seconds no matter how many
mails are in the folder. Specifically Akonadi does not move each mail one time
and then copying it.

-- 
You are receiving this mail because:
You are the assignee for the bug.