[Nepomuk] Nepomuk in 4.13 and beyond
me at vhanda.in
Thu Dec 12 10:46:53 UTC 2013
During the KDE 4.11 cycle Nepomuk reached a maturity level that we were
with, it is reasonably fast, stable, and unless used together with Akonadi it
is no longer the "CPU consumer" it was before. We reached this state after
years of analyzing what was wrong and what could be improved to the
where we no longer think any more improvement is possible only by
The next place where we could seek improvement was the RDF storage. We
been using Virtuoso for about 4 years and it's been a game changer for us
performing way faster than any other of the alternatives we ever used
and more efficiently, but as many of you know (and others suffer) it is not a
storage designed for the desktop and it will never be. Since nothing better
than Virtuoso exists for our use-case, we started to implement our own
RDF storage mechanism (codenamed Vishuoso).
At some point during that progress we took a step back and re-analyzed
problems of the workspace and the current implementation. The problems
- Resource Description Framework (RDF)
The biggest problem with RDF is that it raises the knowledge needed to
contribute to a point where most developers decide to to skip it. After all
these years only a handful of brave developers have worked with it and the
experience hasn't been good.
Then we need something easier to use so we can see a more broad
Additionally, RDF is a very flexible way to store data, it is however not the
most efficient way. Data is generally completely normalized even though it
quite often not required. Eg - One doesn't need to store music file artists
a separate contact. This is great, from a theoretical point of view, but it is
not very useful in practice.
- RDF Storage
There is no existing RDF storage designed to work in a Desktop. Virtuoso
great but it quickly uses hundreds of megabytes of ram and it has its own
share of problems. The other alternative is tracker, but they lack certain
features required in Nepomuk.
- Data duplication
Nepomuk has been used as both a search store and a data store. This
massive data duplication and synchronization problems. In the case of
emails are stored in Akonadi and are then duplicated in virtuoso, and are
duplicated in virtuoso's index. Every time data is changed in Akonadi it has
to be updated in Nepomuk and vice-versa. This results in a process being
responsible just for synchronizing the two stores.
- API Duplication
With the data residing in both Nepomuk and other stores
it isn't always clear which store it should be accessed from. This
results in duplication of APIs. Eg - Using KABC vs accessing contacts from
These problems would still exist even if we had the fastest and most
RDF storage in the world.
At this point it was clear to us that the future was not going to be RDF. The
next thing we did was to analyze our actual needs based on the last 5
Our needs are -
* Full text index for searching
* Data store for simple objects such as tags / ratings / activities / etc
* Relations - Forming relations between different objects. Eg - This "file" is
related to this "activity" or "person".
Each of these problems is independently solvable without RDF.
About 2 months ago we started to draft Baloo , a metadata solution
will cover the bare necessities of each use case we have.
I'd like to avoid getting into the technical details of the implementation in
this thread. Another thread can be started about its different aspects
you've read the basic ideas behind Baloo 
After a month of designing the solution and a month of implementing it,
is working way better than Nepomuk does. So, I'd like to switch to Baloo by
default in 4.13, while keeping Nepomuk in maintenance mode for more
This is not a completely new project as large parts of Baloo code are
derived from Nepomuk and therefore comes with years of testing and real
Baloo was also discussed in PIM Sprint and the PIM developers are happy
completely drop Nepomuk support for 4.13 and move to Baloo. Similarly,
telepathy developers are also working on moving KPeople away from
Migration - There will be an automated migration mechanism for migrating
ratings and comments from Nepomuk to Baloo.
Trying it out?
Developers are welcome to try out Baloo and have a look at the current
code. It's a still a work in progress, but we strongly feel that it is a step
in the right direction.
I'd recommend using Milou  for searching.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Nepomuk