[Kde-pim] Excessive amount of queries

[Aaron J. Seigo]

On Saturday, December 6, 2014 15.07:17 Pablo Sanchez wrote:
> I'm sure everyone knows, it's important to not overtax the resources
> needlessly.  Whether a homegrown data store or a regular old DBMS.

when the store can do in excess of 500k[1] random access record reads[2] per 
second such things begin to matter a bit less. less reads are always better, 
indeed, but if we can make them cheap enough then we can be a bit more relaxed 
about such things.

and by doing all information retrieval through a standard declarative query 
system[3], we can prevent many instances of redundant reads transparently to 
application code. 

right now such logic needs to go into client code, which is impractical and 
quite a demand on application developers, while with a declarative query 
interface we can do quite a bit behind the scenes. for instance, data sets 
between equivalent queries can be implicitly shared. (equivalency is trivially 
computable for such queries.) 

so if we have 20 queries for "how many messages in folder X" happening in 20 
different places in the code, they can easily be made as cheap as one single 
query ... because behind the scenes they will be.

this is why i wanted to get away from SQL and a RDBMS as queries tend not to 
be trivially computable for equivalence and creating a declarative query 
system on top of them is not so straightforward. also, 500k random-access 
reads per second ... :)

unfortunately with the akonadi 1 design, because queries are so expensive and 
the responsibility for query efficiency is mostly left to client code, the 
same problems that are being solved in this thread will almost reappear in 
future development.

[1] on a modern core i7 laptop with a consumer grade SSD
[2] seek to a given record and read just that one record; data set is 50k 
records with 2k+ data in each
[3] "system" is probably an overstatement; i expect it to consist of one 
public class and a couple of private helper classes behind it

-- 
Aaron J. Seigo
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-pim/attachments/20141207/8e3e3cf6/attachment.sig>
-------------- next part --------------
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/