[Kde-pim] Review Request: Add indexing throttling and fixed endless indexing problems

[Volker Krause]

On Sunday 19 February 2012 22:15:26 Volker Krause wrote:
> On Friday 17 February 2012 19:53:42 Volker Krause wrote:
> > On Friday 17 February 2012 09:00:19 Sebastian Trueg wrote:
> > > > On Feb. 16, 2012, 7:45 p.m., Christian Mollekopf wrote:
> > > > > The HighPrio Queue shouldn't ever be throttled ideally, but in view
> > > > > of
> > > > > the current problems it's definitely a reasonable approach. I didn't
> > > > > give a close look yet, but you can ship it from my side. Thanks for
> > > > > the
> > > > > patch.
> > > 
> > > Currently there is no way around throttling the high prio queue. As
> > > stated
> > > above (and as you confirmed in private email) adding a new email account
> > > will result in newItem events for all the emails. That in turn will put
> > > them into the high prio queue.
> > 
> > I'm currently testing this, and it indeed seems to improve indexing
> > considerably. Without throttling in effect my system is now reliably
> > indexing hundreds of mails per minute, without getting stuck with Virtuoso
> > going crazy.
> 
> By now that has changed unfortunately :-(
> 
> The Virtuoso database reached ~800Mb again after two days of indexing all my
> Akonadi data (I deleted it two days ago, to get rid of all the pre-DMS
> cruft in there), and the stuck Virtuoso problem seems to be back. I just
> found it consuming 200% CPU, removing the feeder reduced it to 100%.
> Following Vishesh's  instructions
> (http://vhanda.in/blog/2012/02/virtuoso-going-crazy-/) I got the following
> running query:
> 
> select distinct ?r where { { ?r a
> <http://www.semanticdesktop.org/ontologies/2007/08/15/nao#Tag> . ?v2
> <http://www.semanticdesktop.org/ontologies/2007/08/15/nao#hasTag> ?r . ?v3
> <http://www.semanticdesktop.org/ontologies/2007/08/15/nao#hasTag> ?r . } .
> ?r <http://www.semanticdesktop.org/ontologies/2007/08/15/nao#userVisible>
> ?v1 . FILTER(?v1>0) . } ORDER BY DESC ( count(?v3) ) LIMIT 6
> 
> Closing it did not make it stop though, I had to kill Virtuoso. Restarting
> the feeder afterwards makes that continue normally, with the expected load
> that changes according to the throttling. So, I'm not even sure if this is
> caused by the Akonadi feeder at all. The above mentioned query seems to be
> harmless as well though, run manually it only takes 2.5 secs.

yep, the query seems innocent, just had this again (this time right after 
resume from suspend to ram), removing the feeder reduced the feeder-typical 
load from Virtuoso, but it kept running with an additional 100% I can't 
allocate to anything, there's no active query in Nepomuk.

regards
Volker
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 190 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-pim/attachments/20120220/9c7a1280/attachment.sig>
-------------- next part --------------
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/