[Kde-pim] Review Request 112362: Don't send ItemSearchJob queries through Akonadi

[Christian Mollekopf]

> On Aug. 30, 2013, 7:45 a.m., Volker Krause wrote:
> > Good idea. However, I don't think it solves the actual problem (yet). ItemSearchJob is still an Akonadi::Job, and those are sequentially scheduled by Akonadi::Session. So, you'd still "block" the session while waiting for the results. This probably can be solved by making the job scheduler in Session aware of jobs that don't communicate with the Akonadi server.
> > 
> > Regarding the query language support other than SPARQL, the possible use-case here would be searching on IMAP or LDAP servers via the search interface of resources.
> 
> Christian Mollekopf wrote:
>     IMO the LDAP, IMAP search usecases are very important ones that I'd want to build upon in the future. I see nothing speaking against a shortcut for nepomuk queries though.
>     
>     Since searching resource backends will likely require specifying which resource to search we can either go with nepomuk by default for the search job and using a special constructor for resource search jobs, or we just do as you suggest and make the ItemSearchJob a nepomuk-only search, and create a ServerSideSearchJob when required. I think the latter option is better as the two probably don't have to much in common.
>     
>     So +1 for the idea, but please leave the rest of the server side infrastructure in place.
> 
> Kevin Krammer wrote:
>     Having a different job for server side search sounds a bit problematic, a client would either have to do both jobs or somehow check first if there are resources that can do server side search.
> 
> Christian Mollekopf wrote:
>     I think clients have to explicitly search resources. It's probably not a good idea to search every available backend on every search (maybe it is but it seems expensive). So I was really thinking of clients explicitly specifying which resources to search, especially since the query language is probably specific to the backend.
>     
>     So KAddressbook would i.e. query all ldap resources using the LDAP syntax through akonadi, and kmail all imap resources using whatever syntax IMAP supports.
>     
>     It would of course still be possible to invent a special query language that the backends can convert to LDAP, IMAP, SPARQL or whatever is required, but I think that is not something we should go for from the start, and it will likely always be limited for some edge cases where you'd want the native query language available.
>     
>     So, IMO clients will anyways need to explicitly use server side searches using the right query lanugage, and the currently expected behaviour from the ItemSearchJob is to search nepomuk only, so simply making the status quo the definition of the ItemSearchJob seems reasonable to me.
> 
> Dan Vrátil wrote:
>     I want IMAP/LDAP/... search too, IIRC we even talked about this on Akademy. Obviously supporting IMAP or LDAP search would require going through the Akonadi server. For this reason I don't know whether it makes sense to modify to scheduler to ignore jobs that don't need session - we will need it again once server search is supported. So let's forget about (not) blocking Akonadi session and only take it as "don't bother the server with stuff we can do on client side". The entire search infrastructure will remain, we still need it for persistent search (although that could be probably optimized too - query Nepomuk on client side, then just send a list of matching IDs to the server to create a new virtual collection and link the matching items to it).
>     
>     Regarding the server-side search, I think it should be completely transparent to users what and how it is queried. In future ItemSearchJob should simply query everything (Nepomuk, IMAP, LDAP...) and then just merge results. We don't have to search all resources, the server can easily optimize it by only querying resources that own the collections user wants to search in etc. The server is aware whether Nepomuk is running and can be made aware whether an IMAP resource is disconnected or not and then take all this into account before querying the resources.
>     
>     For the query language, we could probably write a simple query builder (similar to what have in Akonadi for SQL) that would convert into respective query languages, so that it would again hide from users what all sources (and how) are actually queried.

What we want in any case I think is pipelining (executing in parallel) of commands. Search operations are comparably long and should not block the whole session. AFAIK that is the pipelining that is broken in Akonadi::Session. With pipelining working we can indeed execute all kinds of searches (or other jobs that take long), without blocking the whole communication to the akonadi server.
I'm not entirely sure that we should make the search completely transparent but we can see about that when we have the usecases.

I suppose we can conclude we want pipelining to work as that will also "fix" the issue of nepomuk queries which go awry blocking the whole communication (given the server isn't blocked by that either).


- Christian


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://git.reviewboard.kde.org/r/112362/#review38913
-----------------------------------------------------------


On Aug. 29, 2013, 3:50 p.m., Dan Vrátil wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://git.reviewboard.kde.org/r/112362/
> -----------------------------------------------------------
> 
> (Updated Aug. 29, 2013, 3:50 p.m.)
> 
> 
> Review request for KDEPIM-Libraries.
> 
> 
> Description
> -------
> 
> Instead of waiting for Akonadi to talk to Nepomuk and sending us matching items, we query Nepomuk from the ItemSearchJob and then use ItemFetchJob to fetch matching items from Akonadi.
> 
> This prevents blocking Akonadi session in case virtuoso goes nuts and does not respond to queries.
> 
> The only disadvantage of this approach is that we are losing support for anything else but SPARQL and Nepomuk, but we don't use anything else in KDE, so I don't know whether it's really a problem?
> 
> 
> Diffs
> -----
> 
>   akonadi/itemsearchjob.h 87d34b5 
>   akonadi/itemsearchjob.cpp 175c76e 
> 
> Diff: http://git.reviewboard.kde.org/r/112362/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Dan Vrátil
> 
>

_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/