[Kde-pim] Review Request: Add indexing throttling and fixed endless indexing problems

[Volker Krause]

On Friday 17 February 2012 19:53:42 Volker Krause wrote:
> On Friday 17 February 2012 09:00:19 Sebastian Trueg wrote:
> > > On Feb. 16, 2012, 7:45 p.m., Christian Mollekopf wrote:
> > > > The HighPrio Queue shouldn't ever be throttled ideally, but in view of
> > > > the current problems it's definitely a reasonable approach. I didn't
> > > > give a close look yet, but you can ship it from my side. Thanks for
> > > > the
> > > > patch.
> > 
> > Currently there is no way around throttling the high prio queue. As stated
> > above (and as you confirmed in private email) adding a new email account
> > will result in newItem events for all the emails. That in turn will put
> > them into the high prio queue.
> 
> I'm currently testing this, and it indeed seems to improve indexing
> considerably. Without throttling in effect my system is now reliably
> indexing hundreds of mails per minute, without getting stuck with Virtuoso
> going crazy.

By now that has changed unfortunately :-(

The Virtuoso database reached ~800Mb again after two days of indexing all my 
Akonadi data (I deleted it two days ago, to get rid of all the pre-DMS cruft 
in there), and the stuck Virtuoso problem seems to be back. I just found it 
consuming 200% CPU, removing the feeder reduced it to 100%. Following 
Vishesh's  instructions (http://vhanda.in/blog/2012/02/virtuoso-going-crazy-/) 
I got the following running query:

select distinct ?r where { { ?r a 
<http://www.semanticdesktop.org/ontologies/2007/08/15/nao#Tag> . ?v2 
<http://www.semanticdesktop.org/ontologies/2007/08/15/nao#hasTag> ?r . ?v3 
<http://www.semanticdesktop.org/ontologies/2007/08/15/nao#hasTag> ?r . } . ?r 
<http://www.semanticdesktop.org/ontologies/2007/08/15/nao#userVisible> ?v1 . 
FILTER(?v1>0) . } ORDER BY DESC ( count(?v3) ) LIMIT 6

Closing it did not make it stop though, I had to kill Virtuoso. Restarting the 
feeder afterwards makes that continue normally, with the expected load that 
changes according to the throttling. So, I'm not even sure if this is caused 
by the Akonadi feeder at all. The above mentioned query seems to be harmless 
as well though, run manually it only takes 2.5 secs.

regards
Volker
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 190 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-pim/attachments/20120219/13d1d5b9/attachment.sig>
-------------- next part --------------
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/