Nepomuk feature freeze No. 2

[Richard Dale]

On Tuesday 18 September 2007, Sebastian Trüg wrote:
> Hi again,
>
> after the last discussion I could not sleep well and that resulted in
> Soprano now having three interfaces: TCP, Unix sockets, and DBus. The
> performance results are not really what I expected and by far not as
> impressive as I thought.
>
> Task                            | TCP   | Unix Socket | DBus |
> --------------------------------------------------------------
> adding one statement            |    1  |    1        |    2 |
> adding list of 1000 statements  | 1151  | 1207        | 1343 |
> adding 1000 statements          |  914  |  958        | 1080 |
>
> (all values in milliseconds)
>
> So DBus is slower but only marginal. What confuses me here is that the TCP
> interface is faster than the unix socket one. That seems a bid odd.
Do Unix sockets write to a disk file - is that why? I'm pleased to see that 
dbus is reasonably fast apart from slower latency, which is what I expected 
to see. So it looks like a bulk loading feature would be a good idea? When I 
run strigi it looks as though it is sending one message per triple, rather 
than bulking them up - is that right?

> Anyway, I don't think TCP is the right solution for now as it would require
> the introduction of permission handling. And seeing that unix sockets and
> DBus (after all, DBus uses unix sockets, too) do not differ that much it is
> probably best to stick with the well known, reliable DBus interface. After
> all, it will make programming against it from other languages much easier.
Yes, I'm pleased about that - from the point of view of ruby bindings dbus 
will be much better that needing to wrap yet another app specific IPC api.

Because I haven't actually written an app using the nepomuk daemon yet it is 
hard to imagine uses cases, but it seems to me that it would be useful to 
register for dbus signal that the nepomuk daemon would emit on, say when a 
new foaf:person was added to the datastore. Then the UI of your personal 
contacts dbus client or whatever could be updated to show the new person.

> Now, why do I write this mail at all if I want to stick with the DBus
> interface. Well, at the moment Nepomuk in kdelibs uses the Nepomuk
> middleware I implemented some month back when there were plans in the
> Nepomuk project itself for a federation. That would have meant that
> Nepomuk-KDE could have reused services implemented in other languages (most
> likely java since everything in Nepomuk is hacked in java). Now that the
> Nepomuk project dropped the idea of the federation the middleware as it is
> in kdelibs only introduces unnecessary complexity which I would like to
> remove (So far we only have one Nepomuk-KDE service anyway: the RDF
> repository; and there is no reason to not use KDE service technologies
> now.)
> In its place I would like to use the new Soprano DBus interface. it has the
> following advantages:
>
>   * Uses the clean and (IMHO) very nice Soprano 2 API
>   * Is completely iterator-based which means that handling large quantaties
>     becomes cleaner.
>   * We do not duplicate interfaces (Soprano + the one in kdelibs).
>
> Replacing the middleware would mean:
>   * Updating Soprano in kdesupport to version 2 (currently in
>     branches/work/soprano2)
>   * removing kdelibs/nepomuk/middleware entirely
>   * small internal changes to kdelibs/nepomuk/core to make it use Soprano
>   * Change kdebase/runtime/nepomuk/coreservices to a plain KDE service
>     without Nepomuk service registration and update it to Soprano 2
>     including the new Soprano 2 DBus interface
>
> The public API would only change in one position (except for the removal of
> the middleware which no one uses anyway AFAIK):
>   * Nepomuk::ResouceManager::serviceRegistry() removed
>
> Well, that is it I think.
> Your comments please.
I assume we can still get SPARQL results in xml format as you described in 
your last mails? If so, it all sounds really good to me, and I'm looking 
forward to trying it out. Where do the triples actually get stored -  there 
seem to be two triple stores, one in ~/.nepomuk/share/storage/system and 
another one too? The nepomuk daemon has never written anything to 
the .nepomuk one as far as I can see, although it does get created on start 
up.

-- Richard