Still performance issues, this time: Shrinking a folder with archivemail

[Martin Steigerwald]

Martin Steigerwald - 24.03.20, 17:28:23 CET:
> This is with KDEPIM/Akonadi 19.08 on Debian Sid. I know its a bit
> outdated, so if that is really considerably faster and less resource
> hungry with the more recent version, I am happy to learn about that.
> 
> 
> I did
> 
> […]~/.local/share/local-mail/[…]/.Debian.directory> archivemail -D
> "2019-12-31" -o […]/Debian devel-changes-ml
> 
> devel-changes-ml:
>     archived 121966 of 132644 message(s) (964.5MB of 1053.7MB) in
> 238.4 seconds
> 
> cause the folder with that mailing list became really, really slow.
> 
> I am using archivemail since I have completely given up doing this
> within KMail/Akonadi. Last time I tried it was so slow it would not
> finish within hours to come.
> 
> I got all 4 cores of a Sandybridge dual core CPU clogged with
> PostgreSQL processes + akonadi_indexing_agent since more than half an
> hour. Well plus one pxz process to recompress the archivemail archive
> as xz.
> 
> Also it writes truly insane amounts of data to the disk.
> 
> I also cannot do anything in KMail other than typing this mail since
> more than half an hour. It does now show any individual mail or any
> folder contents.
> 
> However I only *deleted* (!) about 120000 mails. The only thing
> Akonadi has to do is to *forget* about them, cause archivemail
> completed the task at hand in less than 5 minutes already.
> 
> I truly believe that such an action should not use this amount of
> resources.
> 
> I will let it sit like this, let's see how long it will be going to
> hog this machine.

I decided too long.

Even after stopping Akonadi Postgres 12.2 was hogged for minutes. It did 
not respond to SIGTERM, so in the end I did SIGKILL, knowing that this 
might make things inconsistent. Since the most recent action was just to 
delete those 120000 mails I thought it might be fine enough.

Before that I measured sizes in ~/.local/share/akonadi. I got:

% du -sch file_db_data db_data search_db
59M     file_db_data
3,1G    db_data
15G     search_db
18G     total

I decided that 15G search_db is really very much for

% ~/.local/share> du -sh local-mail
20G     local-mail

and removed search_db completely.

I then restarted Akonadi and in KMail things appear to be well.

I know at some time the indexing agent might decide to do a full 
indexing run, but for now things are quiet.

For more what I believe more than a decade things like this come up from 
time to time.

In contrast I use Evolution for work cause of having to deal with Office 
365 and it works much better in Evolution. I never needed to bother with 
any database. Never had any serious performance issues caused by the 
mail client. Never had to fiddle around with anything just to make things 
go smoothly.

Anyway, I bet this will be a one time thing and I bet next time I may 
either stop Akonadi before shrinking the folder *or* kill the indexing 
agent before doing so, cause I believe it was the indexing agent 
triggering this insane extra load.

Aside from that things work well enough, so I may just stick with 
Akonadi for longer although part of me says: Use Evolution and be done 
with any issues like that regarding my mail client. I still prefer KMail 
for various reasons, but dealing with Akonadi worsens KMail experience 
quite a bit.

> I hope that someday Akonadi/KDEPIM will have a somewhat *sane*
> performance behavior for things like this.
> 
> Let's see whether it sends out this mail straight away, but I bet that
> it will do.

Thanks,
-- 
Martin