Akonadi dsaster

[Martin Steigerwald]

Am Mittwoch, 12. Oktober 2016, 08:28:52 CEST schrieb Pablo Sanchez:
> On Wed, 12 Oct 2016 13:59:37 +0200, O. Sinclair wrote:
> > To me Martin makes the perfect point: a user should not have to fidget
> > around in config files for database, .rc files, recreate accounts and so
> > on. There is something inherently unstable in a setup that when doing an
> > upgrade of system goes nearly unusable.
> > 
> > I am still trying to avoid recreating my accounts and redownloading my
> > email (I need offline access) but I can see it getting there day by day.
> > And trying to find that way of getting Baloo to make distribution lists
> > work again, preferably without having to recreate them. Again....
> 
> Hi,
> 
> MySQL/MariaDB/PG all make use of the O/S cache.  The only fiddling I
> think would need tweaking is determining if disabling writing in fact
> is fine.  The other proposed tunes I don't think make much
> difference.

AFAIK that is only true with MyISAM. InnoDB partly uses its own caching and 
while I have no measurement, raising the InnoDB buffer pool size has an effect 
that I claim I can perceive subjectively easily enough. It reduces latencies 
in KMail and and I think it also generates less I/O than with a lower value.

> The biggest gain would still be to stop the KDE apps from sending many
> queries per second.  If I recall correctly, it was upwards of 100+
> queries per second.  There was no need for them.

Sure thats very important. Dan/Milian did some work there, but its still too 
much as I read here in this thread.

Also the folder synchronization nonsense IMHO needs to stop. Often enough even 
with maildir Akonadi synchonrizes the folder. Heck its a maildir that I can 
tell Akonadi is *exclusively* used for Akonadi. So if Akonadi stored mail files 
there, they will be there tomorrow. Still Akonadi at least synchronizes a 
folder after every new startup and depending of settings even in between.

With IMAP that can make some sense, but Akonadi also does to much there. I 
made a bug report what happens when you delete several mails on IMAP with 
Dovecot with delete key: It synchonrizes the folder repeatedly, it seems to 
check whether the mails are really gone. Seriously with about 10-20 presses to 
delete key I can trigger "Receiving folder contents" delay displays in KMail. 
And as an user I ask myself WTF?

For me Akonadi isn´t as bad as for others in this thread, but it still has a 
*lot of* performance related issues, mostly generated by generating completely 
*needless* work.

Another examples is moving tens of thousands of mails or a completely folder. 
For Maildir moving a folder would be as easy as: Move the folder in the 
maildir hierarchy, record the change in the database. What does Akonadi do?

1) Create a new folder
2) Move all mails from the old folder to the Akonadi cache
3) Move all mails from the Akonadi cache to the new folder

basically blocking out KMail for even hours *on Dual BTRFS RAID 1*, since 
Akonadi does not seem to have notiong of a background operation. And also 
generating a MySQL load that a database performance engineer would shake his 
or her head about. I found a way that is slightly more effective:

1) Stop Akonadi
2) Move folder with "mv" or so in Maildir hierarchy
3) Start Akonadi.

This still generates a lot of MySQL related work as it seems to upgrade every 
single mail to its new location (or i.e. remove the old one from the db and 
add the new one), but at least the mails do not seem to be moved through the 
Akonadi cache.

Of course I also reported this.

And yes that also makes me belief that current Akonadi is over-designed.


But I do not think that in general a MySQL would be that bad. We used Zimbra 
at work and it was blazingly fast, using MySQL + Lucene and a custom made file 
based store with hashes and deduplication of same mails. So it can be done. 
Yet, Akonadi is not it. Not yet at least.

And still for Zimbra it may be necessary to tune MySQL… although the used 
higher values for InnoDB buffer pull size to begin with.

IMHO Akonadi is supposed to not be fiddled around and currently it isn´t. It is 
my experience that you need to tweak it in order to gain somewhat acceptable 
performance out of it for larger setups.

Thanks,
-- 
Martin