[Kde-pim] Updating persistent search

[Daniel Vrátil]

Hi folks,

I started adding support for agent search (so that we can query for example 
IMAP server when using online IMAP and emails are not indexed in Nepomuk) and 
adding support for search plugins (so that we don't have to copy-paste parts 
of Baloo into Akonadi like with Nepomuk).

Now I'm working on the persistent searches and ran into a problem with 
updating the search folders. If the query contains a match against a header 
that's not indexed by Nepomuk, or when we have online IMAP and there's a match 
against content of the message, we have a problem, as we won't get complete 
results.

I want to avoid having incomplete results, because it makes the entire feature 
look broken and it's confusing to users ("huh, why did my email suddenly 
appear in the search folder after I read it?"). 


Proposed solutions:

A) Don't support persistent search with matches against non-standard headers 
	and body content

PROS: we can guarantee complete results by re-running all queries every time 
an item is added or modified (and optimize it by enqueueing the items and 
running the queries after some timeout, so that we get some reasonable 
performance when fetching many new emails, or when mass-changing flags. With 
Baloo, the queries are reasonably fast and won't cause your computer to self-
ignite)

CONS: it removes a feature and makes the persistent search less useful



B) Don't support persistent search with matches against body content

PROS: see A)

CONS: see A) + requires indexing all headers into Nepomuk, which will make it 
bigger and will slow down the queries. Also I don't know whether Vishesh is 
actually willing to do that.



C) Support persistent search with matches against non-standard headers and 
	body content

PROS: we don't lose any features

CONS: see B) + we have to also use the agent search feature, which requires 
the resource to talk to remote server, getting results and pushing them back 
to Akonadi. This is not just very slow, but also consumes a lot of bandwidth, 
eats some additional CPU and memory and does not work when computer is offline. 
The offline problem makes it the least preferred solution to me, as it would 
cause the results to be incomplete (at least until we are back online, and can 
run all queries again).



D) Don't support updating search folders

PROS: we run the query when the folder is created, which allows us to use even 
the agent-search feature to handle non-standard headers and body search and to 
get complete results

CONS: the search folder will become a snapshot of results in a certain point 
in time


I would personally go with A) or D). Both guarantee complete results while 
keeping the code simple and run-time resources at reasonable level. D) is also 
the current state of the persistent search in stable, because auto-updating 
has been disabled due to updates being to expensive with Nepomuk. It's however 
considered a regression and a temporary measure. If we want to go with D), we 
have to communicate to users somehow that it's no longer a regression, but the 
way things work.



What do you guys think?
Dan


-- 
Daniel Vrátil
KDE Desktop Team
Associate Software Engineer, Red Hat, Inc.

GPG Key: 0xC59D614F6F4AE348
Fingerprint: 4EC1 86E3 C54E 0B39 5FDD B5FB C59D 614F 6F4A E348
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-pim/attachments/20131214/8c708c87/attachment.sig>
-------------- next part --------------
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/