[Kde-pim] Nepomuk feeder queuing problem

[Sebastian Trüg]

On 01/17/2012 12:14 AM, Allen Winter wrote:
> On Monday 16 January 2012 12:41:33 PM Christian Mollekopf wrote:
>> Hey Sebastian,
>>
>> I don't see the problem with this as this timer just kickstarts the
>> processing and can be called as many times as you want. The ItemQueue
>> does the actual feeding of the data. The processing of the ItemQueue is
>> triggered by the processItem() function which simply does nothing if
>> called repeatedly (mRunningJobs > 0). processItem() is also the function
>> which gets called if you call setOnline() or addItem(), but that
>> shouldn't hurt as far as I can see.
>>
>> So there should be at maximum two nepomuk jobs running at a time (one
>> per ItemQueue).
>>
>> Are you sure virtuoso is really going crazy? While indexing virtuoso
>> takes all the cpu power it can get, but that seems normal to me. Also
>> the indexing can take veeery long, so it's quite possible that users
>> think virtuoso just went crazy.
>>
>> Note that some merges of indexed items still fail (I believe it has to
>> do with the same email address appearing several times in an email),
>> maybe this could trigger a problem?
>> I could send you a testitem which triggers the problem in case you wish
>> (it contains private data and is therefore not suitable to be uploaded
>> on a bugtracker).
>>
>> Also it seems that some items get regularly reindexed because some
>> property of an akonadi-item changed (which always triggers a full
>> reindexing). I'm not yet sure why that happens exactly though. This just
>> to say that the feeder can produce similar load to the initial indexing
>> also after the initial indexing has finished.
>>
>> So I think there are lots of improvements to be made regarding
>> performance, but I'm not aware of a problem that overloads nepomuk.
>>
>> Thanks for looking into this.
>>
>> Cheers,
>> Christian
>>
>> On Mon, Jan 16, 2012, at 06:01 PM, Sebastian Trüg wrote:
>>> Hi Christian,
>>>
>>> there seems to be a problem with the Nepomuk feeder in KDEpim. The
>>> reports about virtuoso going crazy pile up and now I ran into the same
>>> problem.
>>> Thus, I had a look at the feeder and found a potential problem:
>>>
>>> You use a single shot timer to continue the indexing. This timer tells
>>> the queue to continue the processing of the items. This is all fine
>>> internally. But both setOnline() and addItem() start the timer without
>>> checking if any job has already been started. Thus, another job will be
>>> started. This can potentially lead to tons of nepomuk jobs running which
>>> could in fact make virtuoso go crazy as there is no real protection
>>> against such an "attack" in Nepomuk.
>>>
>>> Please have a look and tell me if I read the code correctly.
>>>
>>> This is something we should sort out before the final tag.
> 
> Also please try to find some time to fix the searching problems identified
> by Volker and Christian in the thread "Does KMail Searching Work for Anyone?"

I did not find anything. Could you please direct me to the problem.

> Really something else we need to have fixed before 4.8.0 final.
> 
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/