[Akonadi] [Bug 338402] File system cache is inneficient : too many file per directory

[Martin Steigerwald]

https://bugs.kde.org/show_bug.cgi?id=338402

--- Comment #9 from Martin Steigerwald <Martin at Lichtvoll.de> ---
With the SizeTreshold=32768 change I get a nice improvement for my work IMAP
account on the laptop (the one I set to download all mails offline).¹

Before:

ms at merkaba:~/.local/share/akonadi> du -sch db_data/akonadi/* | sort -rh | head
-10
2,8G    insgesamt
2,6G    db_data/akonadi/parttable.ibd
245M    db_data/akonadi/pimitemtable.ibd
13M     db_data/akonadi/pimitemflagrelation.ibd
248K    db_data/akonadi/collectionattributetable.ibd
200K    db_data/akonadi/collectiontable.ibd
136K    db_data/akonadi/tagtable.ibd
120K    db_data/akonadi/tagtypetable.ibd
120K    db_data/akonadi/tagremoteidresourcerelationtable.ibd
120K    db_data/akonadi/tagattributetable.ibd

ms at merkaba:~/.local/share/akonadi> find file_db_data | wc -l
524917


ms at merkaba:~/.local/share/akonadi#130> /usr/bin/time -v du -sch file_db_data
7,0G    file_db_data
7,0G    insgesamt
        Command being timed: "du -sch file_db_data"
        User time (seconds): 2.14
        System time (seconds): 95.93
        Percent of CPU this job got: 29%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 5:35.47
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 33444
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 1
        Minor (reclaiming a frame) page faults: 8079
        Voluntary context switches: 667562
        Involuntary context switches: 60715
        Swaps: 0
        File system inputs: 31509216
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

After the change and akonadictl fsck:

ms at merkaba:~/.local/share/akonadi> find file_db_data | wc -l ;  du -sch
db_data/akonadi/* | sort -rh | head -10
27
7,5G    insgesamt
7,3G    db_data/akonadi/parttable.ibd
245M    db_data/akonadi/pimitemtable.ibd
13M     db_data/akonadi/pimitemflagrelation.ibd
248K    db_data/akonadi/collectionattributetable.ibd
200K    db_data/akonadi/collectiontable.ibd
136K    db_data/akonadi/tagtable.ibd
120K    db_data/akonadi/tagtypetable.ibd
120K    db_data/akonadi/tagremoteidresourcerelationtable.ibd
120K    db_data/akonadi/tagattributetable.ibd

Yep, thats 27 files, instead of >500000 (after just a week of the last fsck,
which reduced to about 500000 files, from 650000+).

After a nice vacuuming I even get:

ms at merkaba:~/.local/share/akonadi> find file_db_data | wc -l ;  du -sch
db_data/akonadi/* | sort -rh | head -10
27
6,5G    insgesamt
6,2G    db_data/akonadi/parttable.ibd
245M    db_data/akonadi/pimitemtable.ibd
13M     db_data/akonadi/pimitemflagrelation.ibd
248K    db_data/akonadi/collectionattributetable.ibd
200K    db_data/akonadi/collectiontable.ibd
136K    db_data/akonadi/tagtable.ibd
120K    db_data/akonadi/tagtypetable.ibd
120K    db_data/akonadi/tagremoteidresourcerelationtable.ibd
120K    db_data/akonadi/tagattributetable.ibd

merkaba:/home/ms/.local/share/akonadi> du -sh file_db_data 
6,5M    file_db_data


I definitely prefer this over the original situation.

Original was 2,8 GiB DB + 7 GiB file_db_local.

Now is 6,5 GiB DB + 6,5 file_db_local and more than 524000 files less to
consider for rsync and our enterprise backup software.

Let's see whether it brings an performance enhancement, but for now I like
this.


[1] Re: Possible akonadi problem?
From: Dmitry Smirnov
Date: Fri, 23 Jan 2015 07:17:13 +1100
Message-id: <10777191.dsSRuLrDof at debstor>

https://lists.debian.org/debian-kde/2015/01/msg00055.html

Thanks,
Martin

-- 
You are receiving this mail because:
You are the assignee for the bug.