[Nepomuk] Question about Nepomuks caching mechanism

[Vishesh Handa]

On Wednesday 28 Aug 2013 22:35:23 Jörg Ehrichs wrote:
> Hi all,
> 
> I have a small question regarding the caching mechanism in the current
> KDE 4.11 Nepomuk implementation.
> 
> In Conquirere I use Nepomuk as my data storage to show and manipulate
> the connections between the nfo:Document and nbib:* Nepomuk resources.
> 
> This works like a charm and is blazing fast when I display available
> as well as new resources. Now Nepomuk added a caching mechanism, while
> I save the retrieved Nepomuk2::Resources to display them in my
> application.
> 
> When I start to add changes and send them via Nepomuk2::setProperty I
> can observe the changes via the resourcewatcher. I do get the correct
> updates via:
> 
> propertyChanged (const Nepomuk2::Resource &resource, const
> Nepomuk2::Types::Property &property, const QVariantList &addedValues,
> const QVariantList &removedValues)
> 
> and I can see the updated property and its value, but the resource is
> still a cached one with its old values, even with several updates I
> won't get any changes in the Nepomuk2::Resources until I restart the
> application.
> 
> I've seen there is a new method .setWatcherEnabled(true) but it does
> not seem to have the desired effect.
> 
> In former versions I could get an updated Nepomuk2:Resource by using a
> 
> QEventLoop loop;
> QTimer::singleShot( 1000, &loop, SLOT(quit()) );
> loop.exec();
> 
> as mentioned in bug: 306108
> this however does not seem to work anymore.

It should work. In fact the tests do seem to pass. I'll take a look at the bug 
report.

> 
> Now the question. How is the design meant to work?
> Is it possible to force the update of Nepomuk resources without
> getting the cached ones? As of now even creating a new
> Nepomuk2::Resource based on the uri returns a cached one.

Right. That is by design. Fixing the bug above should fix this problem as well.

> Or is the Nepomuk storage design to be a fast lookup/search for
> existing data and I should rather copy the data into a local storage
> and update this via the propertyChanged signal from the
> ResourceWatcher?
> 
> Any help with this is very appreciated, as I'm currently lost what the
> "best practice" in such a situation is. I like to avoid keeping my own
> database that pushes data to nepomuk the way akonadi is handling it
> with the mails/events.

The best practice is - it very much depends on your needs.

For casual use - the Resource class is fine. However, using a large number of 
Resource classes does not seem to scale, because each one issues a query 
fetching all the properties. You might not need all the properties. (you 
certainly almost never need the plain text content of the files to be loaded in 
memory for pdf files)

When you have a lot of data - Use sparql queries, fetch the data, store it 
however you want, and use the resource watcher to keep it up to date. When 
you just care about a very limited subset and the additional overhead of the 
Resource class is acceptable, then feel free to use it.

Ideally the Resource class should provide a signal as to when the data has 
been changed, though it currently cannot do so because it is not a QObject. 
KF5 will change this.

Does this adequately answer your question?

> 
> Kind Regards,
> Joerg
> _______________________________________________
> Nepomuk mailing list
> Nepomuk at kde.org
> https://mail.kde.org/mailman/listinfo/nepomuk

-- 
Vishesh Handa