Proposal: Integration of libKMetaData into kdelibs

[Sebastian Trüg]

On Thursday 21 December 2006 22:02, Kevin Ottens wrote:
> Le Lundi 18 Décembre 2006 17:44, Sebastian Trüg a écrit :
> > Let me summarize what libKMetaData is (taken from the libkmetadata
> > documentation):
> > [...]
> >
> > KMetaData's core is designed to allow arbitrary types of meta data, i.e.
> > any resource can be related with any other resource or value by simply
> > naming the relation and providing the value. The power of KMetaData,
> > however, lies in that it provides a class for each type of resource. Each
> > of these classes provide convinience methods to allow a simple handling
> > of the meta data.
>
> Is there a way to make more complex queries? Or is that something planned
> for another library than libKMetaData?

At the moment more complex queries can be done via the query service which 
works directly on the rdf store. I will, however, add more convinience 
methods to the resource classes. Planned for the next days is:

* Reverse properties. For example a method Tag::isTagOf which returns a list
  of resources that are tagged with the tag in question

Traversing the graph deeper than 1 is not yet planned for KMetaData. So far I 
thought this is more of a client task. Maybe it could even be integrated in 
Strigi. Let Strigi create a cache of all the meta data (including the one in 
KMetaData) and infer new links with a weight factor. The deeper we have to go 
down the graph to find a relation the lower the weight factor. That is just 
an idea that I want to follow next year.

> > For an example have a look at the File class.
> > The types of resources and their properties are defined in the The
> > Nepomuk Desktop Ontology.
>
> Just to be sure, is it the ontology.rdf available there?

Yes, it is very very small for now since I write a quick dummy one. We 
definetely have to work on this more. Creating this ontology is a big task in 
Nepomuk but I think it should be influenced by the KDE community and be made 
a freedesktop.org standard which defines the fields of all common meta data 
like id3 tags and so on. New types could easily be added by inheriting from 
the desktop ontology.

> > (To get an even better understanding of KMetaData and how to use it
> > please compile trunk/playground/base/nepomuk-kde and run doxygen on
> > kmetadata/Doxyfile in the build dir.)
>
> I didn't get around to compile it yet because of soprano

I just fixed the dependancies as well as Soprano. Please try again.

> > The proposal is to integrate libKMetaData into kdelibs. At the moment
> > kmetadata has two dependancies:
> > * Soprano (aka QRDF) which can be found in trunk/playground/base/qrdf is
> > used to parse the Nepomuk desktop ontology (which is not complete yet)
> > during build time.
>
> Note that it has some visibility issues, no class are exported in soprano
> which makes the tests linking to fail.

Daniele, the maintainer of Soprano comes from the Java world so we have to cut 
him some slack here. I will deal with the visibility issue right away.

> > * The nepomuk-kde backbone lib is used to communicate to the local meta
> > data store.
> > [* The Nepomuk-KDE core services provide a DBus interface to the soprano
> > meta data store that follows the Nepomuk standard.]
>
> As you already know, I've some concerns on using a RDF based storage
> regarding performances. Some more testing is probably required. What makes
> me ponder is: should we test this before including it in kdelibs? or,
> include the lib in kdelibs first and if performances are not good hope
> it'll improve?

Although I hope it will not be necessary changing the storage solution is very 
simple.

> Of course we can optimize our code, but since it's really tied to RDF, if
> the RDF storage is identified as the bottleneck we'll probably have a hard
> time improving the situation.
>
> In any case, we need a very clear picture on the performance impact on
> using libKMetaData (and what will probably follow) for application
> developers. If it doesn't scale well enough it means that we'll probably be
> careful on how apps use it and how much data can be stored.
>
> Btw, if I find time and can get both soprano and nepomuk-kde (urgh! this
> name... :) ) to build I'm willing to help with such testing.

great. And yes, a new name has to be found soon.

> > The last one is optional since it is a run-time dependancy which later
> > can be replaced by virtually any Nepomuk storage service implementation.
> > However, having a storage service in KDE would probably be a good idea.
>
> Indeed, if I get it right, it's like having Phonon or Solid shipped without
> any backend...
>
> > Please consider libKMetaData as a first step towards a semantic KDE.
> >
> > Next steps could include adding support for meta data of type 1 to
> > libKMetaData, thus wrapping it all under one simple API.
>
> Well, that would need to be done I guess. As a developer, I would find
> weird to use on API for some of the metadata and another one for more
> metadata...

exactly.

> Now I have a few questions and comments about the classes in this library.
>
> I'm not exactly sure if the Ontology class is well named. AFAIU it, it
> allows to access only a part of the ontology, the other part (properties
> cardinality, inheritance tree, etc.) being in the generated classes.

I already noticed that and want to extend it some more. I have planned for 
libKMetaData to support property inheritance this way.
I think the Ontology class should provide all necessary information about the 
ontology. Then the name should be ok, too.

> Moreover, I wonder why you use QString everywhere in this class (and
> probably in others). At quite some places you need URIs, so why not use
> [Q|K]Url? AFAIK they should be suited for this.

In the beginning it was to simplify and because I wanted to stay compatible to 
all the java stuff in Nepomuk. But now there is probably no reason not to 
change that to use KUrl or QUrl.

> Finally, I really ponder if your Variant class should be kept. It would
> probably be more consistent with the rest of kdelibs by using QVariant. If
> the only feature really needed is to store Resource instance in QVariant
> it's probably feasible.

QVariant does not provide the power I need. Variant does three main things:
* inherit QVariant privatly, i.e. no adding of new types, only simple types, 
  Resource, and lists of them.
* automatic list conversion, a Variant containing an int can simply be 
  converted to a QList<int> by Variant::toIntList
* automatic string conversion: toString and toStringList always return usable
  values, independant of the type used in Variant.
So the class is actually necessary and instead of removing ot a rename would 
be better suited. Maybe something like PropertyValue or even Property with an 
added URI.

> Oh and before I forget. Thanks a lot for the good work you've done on this
> topic!
>
> That was my 0.02€.

thanks a lot for your input. Maybe I should get on the QUrl port right away. 
Although I am not sure which one to use... probably KUrl, right?

Cheers,
Sebastian