Scrap baloo?

[Boudhayan Gupta]

Hi,

On 28 September 2016 at 20:33, Christoph Cullmann <cullmann at absint.com> wrote:
> Hi,
>
>> Hi,
>>
>> On 28 September 2016 at 02:36, Christoph Cullmann <cullmann at absint.com> wrote:
>>> any update?
>>
>> Yep. In all the happennings of the week I just forgot to write this email.
>>
>> If Baloo is going to be an integral part of the Plasma experience, do
>> we really want to depend on an external project where we don't have
>> control (and indeed, sentiments may prevent unrestricted contributions
>> based only on merit). This is the political reason why I don't want to
>> depend against Tracker. The technical reason is that it's based on
>> SQLite, which is incredibly slow compared to what we do now.
> I don't see really that it is slow compared to what we do, if you have benchmarks
> for that, I would be pleased to see them.

So would I. You already have Tracker based code, could you spare some
time and run some?

>> At the same time, LMDB needs to be replaced, and fast. I'm building a
>> new KVDB as an university project (it should be able to do 256GB
>> indexes on 32bit machines), and if that doesn't work out there's
>> Sophia (http://sophia.systems/). I'll be evaluating both as a
>> replacement to LMDB.
> Do we really want to maintain a own DB system?
> IMHO that will never work out, all DB systems around need more maintenance power
> than we have.

This is something I'm not sure about. The DB will be build anyway, my
graduation depends on it :D And if I'm going to do something I will do
it well, so it'll be simple and clean.

If it doesn't work out, there's always Sophia to fall back on.

>> Vishesh also wanted to separate out the engine and make it public API
>> (apparently other projects want to make use of it as a general data
>> storage library - and the engine offers fulltext search capabilities
>> and other fancy logical operators that make it particularly
>> attractive. My plan is to move towards that, and eventually also not
>> only index files but also other kinds of objects - contacts, or
>> people, for example.
>>
>> I don't want to move back into the "semantic desktop" idea at all, but
>> I do want some sort of infrastructure that allows for an "action on
>> object" metaphor - file objects can be opened with an application,
>> people objects can be sent mails, and so on.
>>
>> Hope this makes sense.
> I still not see how that should work out, atm, IMHO facts are:
>
> 1) baloo is not maintained

It will, now.

> 2) lmdb will e.g. never work for us on NFS homes and the code needs major overhaul
> to handle errors (which you confirm)

LMDB goes away, either way.

> 3) you said you have "some time" left to maintain it, but you now propose in addition to maintain
> Baloo to write a DB system from scratch, I don't really see that working

I have a personal interest, an academic interest, and now a
KDE-related interest in the KVDB. It *will* work, because I'm the kind
of guy who puts a lot of time and effort into things (maybe even
disproportionately so) into things that genuinely interest me. My
challenge will be to make the codebase so that after I'm done with
this (say in about 5 years or so) it'll be comprehensible to the next
maintainer.

> 4) tracker on the other side is maintained and in use and we can share the index data with GNOME and others
>
> I really doubt that doing the work to remove lmdb, replace it with an "own one" and then starting
> to fix all other issues (like indexer running amok, broken file extractors, ...) will work out if
> we don't clone some more people.
>
> But that is only my opinion.

*Sigh*

I don't want to take the easy way out here. Half the fun in KDE is
doing crazy things and seeing your baby work. That's the entire
motivation for being here.

And right now I'm volunteering to do this.

-- Boudhayan