[Nepomuk] Performance issues in KDE 4.8.1

Fri Mar 9 19:12:29 UTC 2012

On Fri, Mar 9, 2012 at 7:52 PM, Ignacio Serantes <kde at aynoa.net> wrote:

>
> On Fri, Mar 9, 2012 at 9:05 AM, Vishesh Handa <me at vhanda.in> wrote:
>
>>
>>
>> On Fri, Mar 9, 2012 at 1:25 PM, Ignacio Serantes <kde at aynoa.net> wrote:
>>
>>> Hi,
>>>
>>> I'm not sure but seems like creating a resource with Nepomuk.Resource()
>>> is slow than before and needs more CPU. This is a point that
>>> probably Ehrichs could confirm better than me.
>>>
>>
>> Yup. It should be slower, we're now doing error checking. I'm not sure
>> about it needing more CPU than before, though.
>>
>
> If you could write code that do error checking and logical analysis
> without requiring more CPU you will don't have problems to found a job in
> the future. But, I wonder why are you checking errors when you are only
> reading data?
>

We aren't. The data reading code is exactly the same.

>
>
>>
>> An about the synchronous has it's advantages where there is a sort in
>>> client side, there is a lot of resources, you are handling correctly the
>>> fetch on demand and you are connecting to an internet server but, if  there
>>> is performance issues returning 144 records from a local database server
>>> because the operation is synchronous, I think we have a little problem. In
>>> fact with so few records probably asynchronous operation will be slow in
>>> total time ;).
>>>
>>
>> Uhh. I lost you. Isn't 144 records a lot?
>>
>
> No, it's nothing, there are practically 0 records.
>
>
>>
>> What happens with the synchronous version is -
>>
>> 1. One blocking query to get all the resource uris of type nao:Tag
>> 2. One query for each nao:Tag in order to load *all* of its properties,
>> and then store them in a cache.
>>
>
> I don't follow you, it's the same you are using synchronous or
> asynchronous communication, you need to do a query to obtain the result set
> you are looking for. For example in the case of tags a good one will be:
>
> SELECT ?uri ?identifier ?prefLabel
> WHERE {
>   ?uri rdf:type nao:Tag .
>   ?uri nao:identifier ?identifier .
>   ?uri nao:userVisible 1 .
>   OPTIONAL { ?uri nao:prefLabel . } .
> }
>
> because you are reading three important fields with only one query and,
> off course, LIMIT and OFFSET are your friends. The method used to transmit
> this information could be synchronous or asynchronous :?.
>

Sorry. I should have been clearer.

Here is what is happening right now -

1. One synchronous blocking query is executed to get all the tag uris,
which are of the form nepomuk:/res/some-uuid
2. For each uri
    One synchronous blocking query is performed which is something like -
    select ?p ?o where { <nepomuk:/res/the-uuid> ?p ?o. }
3. This all is stored in the cache

That's why it is slow.

>
>> Considering that we do not need all of the properties and we only require
>> a label. One query would be a lot better, even if it is synchronous.
>>
>>
>>>
>>> How many queries are required to create a resource? Is cache working as
>>> expected? I know there is a cache because I have issues, expected cache
>>> issues :), developing nepoogle so it's still working as expected? Obviously
>>> there is something different in the last version of KDE.
>>>
>>
>> What kind of cache issues are you experiencing? Cause with 4.8.1 we have
>> added extra code to make sure that the cache is always up to date.
>>
>
> As I wrote, expected cache issues basically related with filex:/ protocol
> when I plug and unplug an external HD. In this case cache results are
> outdated and I found this a logical behavior easy to solve using
> Nepomuk.ResourceManager.instance().clearCache().
>

Hmm. We didn't think of that.

I'll take a look at it.

>
>
>>
>>
>>>
>>> On Fri, Mar 9, 2012 at 6:52 AM, Vishesh Handa <me at vhanda.in> wrote:
>>>
>>>>
>>>>
>>>> On Fri, Mar 9, 2012 at 1:39 AM, Ignacio Serantes <kde at aynoa.net> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Just installed KDE 4.8.1 I found a severe performance issue with my
>>>>> plasmoid Drop2TagIcon.
>>>>>
>>>>> Some investigation reveals that the problem is the call to
>>>>> Nepomuk.Tag.allTags() and, as I have 5 Drop2TagIcon in each of my four
>>>>> activities, the final result is that Plasma was totally frozen.
>>>>>
>>>>> Sadly this seems not restricted to Python, in irc Jörg Ehrichs has
>>>>> similar issues with Conquirere and a few minutes ago, when I'm trying to
>>>>> tag files using Dolphin in the old fashion way, I discover that the problem
>>>>> is here too.up
>>>>>
>>>>
>>>> Yup. Nepomuk::Tag::allTags() is synchronous, and ideally, shouldn't be
>>>> used.
>>>>
>>>> I've noticed the performance impact as well.
>>>>
>>>>
>>>>> More information:
>>>>> - I have only 144 tags.
>>>>> - Over 7-8 seconds passed between I click in Dolphin "Add Tags..." and
>>>>> the dialog is displayed.
>>>>> - Meanwhile over 33% of my dual core CPU is used by the
>>>>> nepomukservicestub that launches virtuoso.
>>>>> - The next query:
>>>>>
>>>>> SELECT ?uri ?label
>>>>>
>>>>> WHERE {
>>>>>
>>>>> ?uri rdf:type nao:Tag .
>>>>>
>>>>> OPTIONAL { ?uri nao:prefLabel ?label . } .
>>>>>
>>>>> } ORDER BY ?label
>>>>>
>>>>> took over 150-170 milliseconds using NepSak and consuming 6% for an
>>>>> instant.
>>>>>
>>>>
>>>> That seems decent. I was going to use the QueryLibrary, but maybe I
>>>> could directly use sparql.
>>>>
>>>>
>>>>>
>>>>> --
>>>>> Best wishes,
>>>>> Ignacio
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Nepomuk mailing list
>>>>> Nepomuk at kde.org
>>>>> https://mail.kde.org/mailman/listinfo/nepomuk
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Best wishes,
>>> Ignacio
>>>
>>>
>>>
>>> _______________________________________________
>>> Nepomuk mailing list
>>> Nepomuk at kde.org
>>> https://mail.kde.org/mailman/listinfo/nepomuk
>>>
>>>
>>
>
>
> --
> Best wishes,
> Ignacio
>
>
>
> _______________________________________________
> Nepomuk mailing list
> Nepomuk at kde.org
> https://mail.kde.org/mailman/listinfo/nepomuk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/nepomuk/attachments/20120310/708774ef/attachment-0001.html>