[Kde-pim] Review Request: Add indexing throttling and fixed endless indexing problems

[Sebastian Trüg]

On 02/19/2012 10:15 PM, Volker Krause wrote:
> On Friday 17 February 2012 19:53:42 Volker Krause wrote:
>> On Friday 17 February 2012 09:00:19 Sebastian Trueg wrote:
>>>> On Feb. 16, 2012, 7:45 p.m., Christian Mollekopf wrote:
>>>>> The HighPrio Queue shouldn't ever be throttled ideally, but in view of
>>>>> the current problems it's definitely a reasonable approach. I didn't
>>>>> give a close look yet, but you can ship it from my side. Thanks for
>>>>> the
>>>>> patch.
>>>
>>> Currently there is no way around throttling the high prio queue. As stated
>>> above (and as you confirmed in private email) adding a new email account
>>> will result in newItem events for all the emails. That in turn will put
>>> them into the high prio queue.
>>
>> I'm currently testing this, and it indeed seems to improve indexing
>> considerably. Without throttling in effect my system is now reliably
>> indexing hundreds of mails per minute, without getting stuck with Virtuoso
>> going crazy.
> 
> By now that has changed unfortunately :-(
> 
> The Virtuoso database reached ~800Mb again after two days of indexing all my 
> Akonadi data (I deleted it two days ago, to get rid of all the pre-DMS cruft 
> in there), and the stuck Virtuoso problem seems to be back. I just found it 
> consuming 200% CPU, removing the feeder reduced it to 100%. Following 
> Vishesh's  instructions (http://vhanda.in/blog/2012/02/virtuoso-going-crazy-/) 
> I got the following running query:
> 
> select distinct ?r where { { ?r a 
> <http://www.semanticdesktop.org/ontologies/2007/08/15/nao#Tag> . ?v2 
> <http://www.semanticdesktop.org/ontologies/2007/08/15/nao#hasTag> ?r . ?v3 
> <http://www.semanticdesktop.org/ontologies/2007/08/15/nao#hasTag> ?r . } . ?r 
> <http://www.semanticdesktop.org/ontologies/2007/08/15/nao#userVisible> ?v1 . 
> FILTER(?v1>0) . } ORDER BY DESC ( count(?v3) ) LIMIT 6

This is a very pointless query. Any idea where it comes from?

> Closing it did not make it stop though, I had to kill Virtuoso. Restarting the 
> feeder afterwards makes that continue normally, with the expected load that 
> changes according to the throttling. So, I'm not even sure if this is caused 
> by the Akonadi feeder at all. The above mentioned query seems to be harmless 
> as well though, run manually it only takes 2.5 secs.
> 
> regards
> Volker
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/