martin at lichtvoll.de
Wed Jan 28 13:34:33 GMT 2015
Am Mittwoch, 28. Januar 2015, 07:53:12 schrieb Pablo Sanchez:
> As a Database and Performance engineer, I've investigated akonadi.
> What I found at the database level is an excessive number of queries
> (IIRC, 1000's per second) which because all the data was in the DBMS'
> cache, manifested itself in 100% CPU utilization.
Did you test this with Akonadi git
c733429f4fa9696fb027ddc946e54f6bbb68deaf (branch 1.13)?
It contains gems like:
Avoid ridiculous amount of SQL queries by caching PartTypes
Implement cache for CollectionStatistics to significantly reduce amount of
Avoid repeated calls to PimItem::flags() and PimItem::tags()
Avoid recursive collection listing in SearchHelper
> I used a VM on a 8p Host O/S. The more processors I gave the VM, the
> more queries per second executed. Which, on the face of it, one might
> say, great, our application is scaling well. However, whenever I
> engage in this type of analysis, the question I always ask is what's
> the reason for the query which is our top consumer.
> In this particular case, what I found was one of the top consuming
> queries was checking for new email. 1000's of times per second? This
> makes no sense. Even with a super high-speed connection, who cares!
> Daniel (sustaining engineer?) rolled in some changes to mitigate the
> incessant queries. I don't know which version of Akonadi has these
Yes, see above :)
And: It *does* help. I use it.
Still I think Akonadi caches too much and still has a lot of other issues.
I posted quite some bug report numbers here already.
> IMO, the concept of Akonadi is a good one but the implementation
> (1000's of queries per second?) needs refactoring. In the spirit of
> OSS, there will be /Akonadi Next/ .... with no disrespect intended,
> I'd prefer to see Akonadi fixed than to create a new set of potential
I agree that the concept behind Akonadi is basically a good one.
But I also see the ideas for Akonadi Next as valid. While I know that
using MySQL for handling the metadata of a ton of mails is possible as a
groupware server like Zimbra shows by being able to access a folder with
>480000 mails (thats right!) in the Web gui withing 5-10 seconds, I think
that Akonadi has implementation issues. Sure, Zimbra tricks it, by just
showing the first several thousand mails in the beginning and loaded more
when you scroll down, but also the Lucense search index works
marvellously. So, it *is* possible. This was with a VM with 12 GiB of RAM,
but not just for one user, but for 15-50 users at once.
I think its possible to fix quite some of these implementation issues, but
I think its multi process design leaves much room for synchronisation
issues and may be difficult to get right. Also the scheduler within Akonadi
is quite broke IMHO in blocking interactive user requests for background
tasks, leaving KMail blocked for minutes. And it caches too much, way too
much. 75 GiB in file_db_data is just too much IMHO.
I think Akonadi is in a similar situation as Nepomuk was after Vishesh´s
optimization work. It often basically worked, but Baloo shows at what
performance cost. Vishesh got to the point that he thought not being able
to optimize it much more without replacing it.
So I think it may make sense to *simplify* it big time, while keeping the
basic advantages. Whether it then uses MySQL or not, well… – while I do
not like starting a MySQL for each user all that much – I don´t care that
much on the other hand, if it works. But I do think the ideas of Akonadi
Next make sense. And it may even make sense to use something else than a
As does making Akonadi more robust and speedy, cause Akonadi Next will
likely take some time.
- robustness (! very important, Akonadi lacks here, big time)
- simplicity and introspectability in case of issues
while keeping the advantages of Akonadi was to provide (but didn´t fully
so far), like background processing *without* blocking apps, is fine with
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7
KDE PIM users mailing list
Subscription management: https://mail.kde.org/mailman/listinfo/kdepim-users
More information about the kdepim-users