[Kde-pim] Excessive amount of queries

[Aaron J. Seigo]

On Sunday, December 7, 2014 07.56:10 Pablo Sanchez wrote:
> consultant perspective.  Sometimes people forget to ask those questions:
>   do we really need 500,000 IOPS to support our application for a single

not IOPs. query round trips. 

that's 500,000 individual "give me record X" queries from the client being 
fulfilled from a 50,000 row dataset with rows being a little over 2k in size 
each and whole rows being returned.

can MySQL post those kinds of numbers, client->server->client, on a commodity 
laptop?

saturating available disk IOPs is doable with most reasonable RDBMSs, but that 
number is largely uninteresting. it's the query turn around time. and no, we 
probably don't need 500k queries/s, but given that given each query takes on 
average 1/500,000th of a second it means we can get away with querying and not 
worrying so much about it because the latency is so low.

... I'm pretty sure we can't get those #s from MySQL.

... and with the key/value store we are working with right now it gets even 
better: it memory maps results and the buffer structure we are using reads 
from memory mapped data segments so that there is close to zero copy for 
results. Forget about cross-process data transfer costs, there is as close to 
zero *in-process* transfer costs as well.

This is ignoring other benefits like lower memory usage, not being able to 
share identical-query datasets easily, not having to deal with schema updates, 
... ... ... from a purely performance perspective, going to a key/value store 
offers improvements that are hard to beat, improvements that are likely to 
render things like maintaining a query result cache, and all the complexity 
that comes with such things, unnecessary.

Which comes back to one of the biggest challenges with Akonadi today: to use 
it effectively means ever increasing code complexity on top of it .. and that 
complexity means more maintenance cost and in some places of the code a 
"fossilization" where changes become increasingly difficult due to pre-
existing complexity.

-- 
Aaron J. Seigo
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-pim/attachments/20141207/96cc9e27/attachment.sig>
-------------- next part --------------
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/