[kdepim-users] Work-around to issues with Akonadi file based caching (was: Re: rant)

[Martin Steigerwald]

Am Freitag, 27. Februar 2015, 09:58:04 schrieb Martin Steigerwald:
> Hakan,
> 
> Am Sonntag, 15. Februar 2015, 17:19:45 schrieb E. Hakan Duran:
> > On Friday, February 13, 2015 10:34:28 AM Martin Steigerwald wrote:
> > > ...
> > > The SizeThreshold thing helped me with that. But have caution: This
> > > is
> > > no official recommendation of Akonadi developers. Still, it gave me
> > > a
> > > good improvement with performance and latency *and* seemed to have
> > > solved the insane amount of file and data in file_db_data issue for
> > > me. Or at least it worked around it successfully.
> > >
> > >...
> > 
> > Thank you once again Martin. I made the changes you recommended. I
> > think it feels a little more responsive now, but I have to use it a
> > few more days to make a more objective call.
> > 
> > I still notice a growing file_db_data folder, proportional to the time
> > akonadi is left on. I am considering to delete everything and start
> > from scratch but I have so many accounts (e-mail, personal and work
> > calendars, etc.) it feels like a waste of my time at the moment.
> > Instead, I will keep turning akonadi off as soon as I check and reply
> > to my emails, and then run fsck afterwards to get rid of the mess it
> > generated until I have more time for the revamp or I am truly pissed
> > 
> > :).
> 
> so far this change works for me.
> 
> Several POP3 accounts, one very large, all in one maildir, and a 30-day
> limited IMAP account:
> 
> martin at merkaba:~/.local/share/akonadi> find file_db_data | wc -l
> 32
> 
> martin at merkaba:~/.local/share/akonadi#1> du -sh file_db_data
> 4,3M    file_db_data
> 
> Very large IMAP account from work:
> 
> ms at merkaba:~/.local/share/akonadi> find file_db_data | wc -l
> 2376
> ms at merkaba:~/.local/share/akonadi> du -sh file_db_data
> 663M    file_db_data
> 
> 
> Hmmm, yikes. Its large in there, but at least not very many files. And
> well, it seems that are mails with larger attachment that Akonadi caches
> there:
> 
> ms at merkaba:~/.local/share/akonadi> du -sh file_db_data/* | sort -rh |
> head 11M     file_db_data/4735240_r1
> 11M     file_db_data/4735240_r0
> 11M     file_db_data/4735239_r1
> 11M     file_db_data/4735239_r0
> 9,4M    file_db_data/4734016_r1
> 9,4M    file_db_data/4734016_r0
> 9,3M    file_db_data/4731257_r0
> 8,6M    file_db_data/4731853_r1
> 8,6M    file_db_data/4731853_r0
> 7,2M    file_db_data/4732214_r1
> 
> And as I think I set the account to offline caching that even makes
> sense.
> 
> 
> Well all it comes down to is: If you have lots of mails that are larger
> than 32 KiB the change won´t help. You may try with an even larger
> SizeThreshhold, but as I did never test it and no of no one else who
> did, I can´t say whether it would work. Also I don´t know the maximum
> limit for SizeThreshold, I don´t know the limit of MySQL for storing
> these blobs in the database.
> 
> All I see here is that SizeThreshold=32768 works okay here.
> 
> 
> These figures are weeks after the change and I didn´t do any manual
> optimization aka akonadictl fsck or akonadictl vacuum. I may try it on
> the work account next week to see whether Akonadi again lost track of
> some files.

This still seems to work for me:

Since my work-around with SizeThreshold=32768 on my private setup:

martin at merkaba:~/.local/share/akonadi> find file_db_data | wc -l
33
martin at merkaba:~/.local/share/akonadi> du -sh file_db_data 
4,4M    file_db_data
martin at merkaba:~/.local/share/akonadi> du -sh file_db_data/* | sort -rh | 
head -10
896K    file_db_data/2815963_r0
584K    file_db_data/2630687_r0
452K    file_db_data/2630658_r0
448K    file_db_data/2630655_r0
368K    file_db_data/2488539_r0
164K    file_db_data/2758167_r0
152K    file_db_data/2488220_r0
120K    file_db_data/2488152_r0
104K    file_db_data/2565672_r0
100K    file_db_data/2943194_r0


And on my laptop work setup:

ms at merkaba:~/.local/share/akonadi> find file_db_data | wc -l
2442

ms at merkaba:~/.local/share/akonadi> du -sh file_db_data 
703M    file_db_data

still quite much, but I set the account to offline caching and I think it 
cached mails with larger attachments:

ms at merkaba:~/.local/share/akonadi> du -sh file_db_data/* | sort -rh | head 
-10
11M     file_db_data/4735240_r1
11M     file_db_data/4735240_r0
11M     file_db_data/4735239_r1
11M     file_db_data/4735239_r0
9,4M    file_db_data/4734016_r1
9,4M    file_db_data/4734016_r0
9,3M    file_db_data/4731257_r0
8,6M    file_db_data/4731853_r2
8,6M    file_db_data/4731853_r1
8,6M    file_db_data/4731853_r0

Ciao,
-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7
_______________________________________________
KDE PIM users mailing list
Subscription management: https://mail.kde.org/mailman/listinfo/kdepim-users