[Akonadi] [Bug 319175] New: moving a folder structure from a mixed maildir structure to a pure maildir structure produces garbage in target folders

Wed May 1 14:52:30 BST 2013

https://bugs.kde.org/show_bug.cgi?id=319175

            Bug ID: 319175
           Summary: moving a folder structure from a mixed maildir
                    structure to a pure maildir structure produces garbage
                    in target folders
    Classification: Unclassified
           Product: Akonadi
           Version: 4.10
          Platform: Other
                OS: Linux
            Status: UNCONFIRMED
          Severity: critical
          Priority: NOR
         Component: server
          Assignee: kdepim-bugs at kde.org
          Reporter: Martin at Lichtvoll.de

After going through:

1) Create one maildir resource for each maildir main directory of each POP3
account, importing old filter rules, having the migrator migrate the POP3
accounts and other stuff from old configuration. => Mail losses (was on BTRFS
as well)

2) Have the migrator do everything with ~/Mail in place. Strange folder
ordering in KMail, scary popups (I didn´t note these, went away to quickly)

 3) This morning: Import old mails via KMailCVT. KMailCVT completed within a
hour. But then Akonadi got stuck what seemed to be a write loop into
changes.dat file. I restarted it after it has written more than 450 GB(!) which
is about 1,5 times the amount of the capacity of the SSD. Mail unread/read
status not imported correctly.

(bugs reported here)

I tried:

 4) Now: Import via pointing a mixed maildir resource at ~/Mail, moving over
folders
[15:06:23] <helios21> to automatically pre-created pure Local Folders maildir
resource

After moving a small folder with two subfolders from the mixedmaildir pointing
to ~/Mail to the local folders pure maildir successfully I tried a bigger one
with about 168000 mails. It took at least half a hour, maybe a hour till
Akonadi actually created the target maildir folder. Then more and more mail
file appeared there. More than in the source. But most were empty one-byte
sized files just containing a linefeed.

Reproducible: Didn't try

Steps to Reproduce:
1. Setup KMail 2 from scratch
2. Have a mixedmaildir resource point to an old (maildir only) KMail-1 ~/Mail
directory.
3. Let it "Synchronize All"
4. Move over a bigger folder with some subfolders to the pre-created pure
maildir local folders resource
Actual Results:  
Source:
martin at merkaba:/mnt/home-zeit/martin-Mail-2013-05-01/.Lichtvoll.directory/.Linux.directory>
find -type f | wc -l
168299
martin at merkaba:/mnt/home-zeit/martin-Mail-2013-05-01/.Lichtvoll.directory/.Linux.directory>
find -type d | wc -l
154

martin at merkaba:/mnt/home-zeit/martin-Mail-2013-05-01/.Lichtvoll.directory/.Linux.directory/kernel-ml>
find cur -type f | wc -l
42071
martin at merkaba:/mnt/home-zeit/martin-Mail-2013-05-01/.Lichtvoll.directory/.Linux.directory/kernel-ml>
find cur -type f ! -size 1 | wc -l
42071
martin at merkaba:/mnt/home-zeit/martin-Mail-2013-05-01/.Lichtvoll.directory/.Linux.directory/kernel-ml>
find cur -type f -size 1 | wc -l 
0

(including cur/new and so one folders)

Target:

martin at merkaba:~/.local/share/local-mail/.Lichtvoll.directory/.Linux.directory>
find -type f | wc -l
304006
martin at merkaba:~/.local/share/local-mail/.Lichtvoll.directory/.Linux.directory>
find -type d | wc -l
150

martin at merkaba:~/.local/share/local-mail/.Lichtvoll.directory/.Linux.directory#1>
find -type f ! -size 1 | grep -v xfs-ml
martin at merkaba:~/.local/share/local-mail/.Lichtvoll.directory/.Linux.directory#1>
find -size 1 | wc -l          
292456

martin at merkaba:~/.local/share/local-mail/.Lichtvoll.directory/.Linux.directory>
find -type f ! -size 1 -ls | head
1475802    4 -rw-r--r--   1 martin   martin       3114 Mai  1 15:04
./xfs-ml/new/1367413494.R571.merkaba
1482572    4 -rw-r--r--   1 martin   martin       3700 Mai  1 15:06
./xfs-ml/new/1367413573.R557.merkaba
1480531    8 -rw-r--r--   1 martin   martin       6771 Mai  1 15:05
./xfs-ml/new/1367413552.R539.merkaba
1479024   12 -rw-r--r--   1 martin   martin       8950 Mai  1 15:05
./xfs-ml/new/1367413535.R248.merkaba
1477745   12 -rw-r--r--   1 martin   martin      11310 Mai  1 15:05
./xfs-ml/new/1367413518.R446.merkaba
1483894   16 -rw-r--r--   1 martin   martin      13906 Mai  1 15:06
./xfs-ml/new/1367413594.R276.merkaba
1479049    8 -rw-r--r--   1 martin   martin       7755 Mai  1 15:05
./xfs-ml/new/1367413535.R726.merkaba
1484993    8 -rw-r--r--   1 martin   martin       4452 Mai  1 15:06
./xfs-ml/new/1367413613.R453.merkaba
1485314    8 -rw-r--r--   1 martin   martin       6755 Mai  1 15:06
./xfs-ml/new/1367413619.R134.merkaba
1486609    8 -rw-r--r--   1 martin   martin       6410 Mai  1 15:07
./xfs-ml/new/1367413651.R689.merkaba

martin at merkaba:~/.local/share/local-mail/.Lichtvoll.directory/.Linux.directory/kernel-ml/new>
hd 1367412411.R104.merkaba
00000000  0a                                                |.|
00000001

martin at merkaba:~/.local/share/local-mail/.Lichtvoll.directory/.Linux.directory/kernel-ml/new>
find -type f| wc -l 
84142
martin at merkaba:~/.local/share/local-mail/.Lichtvoll.directory/.Linux.directory/kernel-ml/new>
find -size 1 | wc -l
84142

Expected Results:  
An *exact* copy of the mail files from the source (albeit with different
filenames)

Debian Sid with:

martin at merkaba:~> apt-show-versions | egrep "(kmail|akonadi|mysql)"
akonadi-backend-mysql/experimental uptodate 1.9.1-2
akonadi-backend-postgresql/experimental uptodate 1.9.1-2
akonadi-backend-sqlite/experimental uptodate 1.9.1-2
akonadi-server/experimental uptodate 1.9.1-2
akonadiconsole/experimental uptodate 4:4.10.2-2
kmail/experimental uptodate 4:4.10.2-2
libakonadi-calendar4/experimental uptodate 4:4.10.2-2
libakonadi-contact4/experimental uptodate 4:4.10.2-2
libakonadi-kabc4/experimental uptodate 4:4.10.2-2
libakonadi-kcal4/experimental uptodate 4:4.10.2-2
libakonadi-kde4/experimental uptodate 4:4.10.2-2
libakonadi-kmime4/experimental uptodate 4:4.10.2-2
libakonadi-notes4/experimental uptodate 4:4.10.2-2
libakonadiprotocolinternals1/experimental uptodate 1.9.1-2
libdbd-mysql-perl/sid uptodate 4.021-1+b1
libmysqlclient-dev/sid uptodate 5.5.30+dfsg-1.1
libmysqlclient18/sid uptodate 5.5.30+dfsg-1.1
libqt4-sql-mysql/sid uptodate 4:4.8.2+dfsg-11
mysql-client/sid uptodate 5.5.30+dfsg-1.1
mysql-client-5.5/sid uptodate 5.5.30+dfsg-1.1
mysql-common/sid uptodate 5.5.30+dfsg-1.1
mysql-server-core-5.5/sid uptodate 5.5.30+dfsg-1.1
python-mysqldb/sid uptodate 1.2.3-2

All Akonadi data located on Ext4.

-- 
You are receiving this mail because:
You are the assignee for the bug.