[Nepomuk] Nepomuk::ResourceWatcher: Performance issues

[Peter Penz]

Hi,

As mentioned in my mail 2 days ago I'm using Nepomuk's ResourceWatcher 
to be able to reflect resource/property-changes in Dolphin's view.

It works quite fine already but I observed a few implementation issues 
in ResourceWatcher that affect the performance in a bad way... I guess 
it should be no big deal to fix them, however please let me explain 
first how I use the ResourceWatcher in Dolphin:

Dolphin resolves meta-data of directory-items step by step to prevent 
that the user-interface gets blocked. So when opening a directory with 
lets say 20000 items the ResourceWatcher gets used like this:

1. After receiving the 20000 items from KDirLister an instance of 
ResourceWatcher is created and ResourceWatcher::start() is invoked.

2. Asynchronously resolve the metadata of item 1:
    - Read the meta-data and apply it to the Dolphin-model
    - Add item 1 as resource to the ResourceWatcher 
(ResourceWatcher::addResource()).

3. When entering the next event-loop resolve the metadata of item 2
    ...

4. Resolve the metadata of item 3
    ...

5. ... and so on (20000 times)

Now to the problems:

- Invoking ResourceWatcher::start() in step 1 without added resources 
blocks the process (at least in my environment) for a few seconds. The 
problem is gone if I invoke start() after at least one resource has been 
added.

- Invoking ResourceWatcher::addResource() after invoking 
ResourceWatcher::start() has no effect. As workaround before adding a 
resource I needed to invoke ResourceWatcher::stop(), add the resource 
and invoke ResourceWatcher::start() again. The problem with this 
workaround is the performance: Calling stop() clears e.g. 15000 items 
and calling start() after adding one resource adds 15001 items again. In 
the case of the directory with 20000 this might take up a few seconds 
for each (!) of the items in the upper-range.

Please note that calling ResourceWatcher::start() after resolving the 
metadata of the 20000 items is way too late, the data can already be 
changed during the resolving. Also adding 20000 resources in one step 
takes around 110 seconds here :-/

 From a quick look at the implementation I guess it should not be too 
tricky to let the ResourceWatcher internally be aware whether it has 
been started already or not and to consider this when adding a new 
resource without doing an expensive stop/start call.

It would be great if this could be fixed by the Nepomuk-team but I'm 
aware that your time is limited. So my question: Should I submit a 
bug-report and is it realistic that you can fix this before 4.9.0? Or 
should I try provide a patch?  My time is also limited and I'd prefer 
the "Nepomuk-team-fixes-it"-option, but without a fix this would be a 
kind of showstopper from a Dolphin perspective and... - well, I'd 
provide a patch if you cannot work on this ;-)

Still crossing my fingers that you say "no big deal, easy fixable"...

Thanks,
Peter