Fwd: Fwd: Scrap baloo?

[Christoph Cullmann]

FYI
----- Weitergeleitete Mail -----
Von: Howard Chu <hyc at symas.com>
An: cullmann at absint.com
Gesendet: Thu, 15 Sep 2016 03:22:12 +0200 (CEST)
Betreff: Fwd: Scrap baloo?

> Hi,

Hi, FYI
>
> after looking a bit more at the code, I think there are ATM a lot of things
> that need fixing:
>
> 1) 32-bit system: I see no fix, > 1GB of index and baloo + all baloo using
> applications fail

Compile LMDB with -DMDB_VL32 to allow using larger-than-32bit DBs on 32bit 
systems.

>   see bugs like https://bugs.kde.org/show_bug.cgi?id=356114 here we have the
> 5GB limit, which is now raised
>   for 64-bit, but not for 32-bit
>
> 2) Larger filesystems: unfortunately one decided to ignore the upper 32-bit of
> the inodes

Gross.
>
> 3) No error handling of most lmdb faults (like already mentioned)

Sounds like quite sloppy coding practice. Unfortunate.
>
> 4) No error handling for any data corruption: e.g. many places will just
> endless loop or malloc, like
>   DocumentUrlDB::get(quint64 docId) (we have bugs for that)
>
> 5) lmdb locking issues: crash one read-write process => all other things stall
> (or crash because of 3+4)

Update to a recent LMDB with robust mutex support. (Added in 0.9.17, 2015-11-30)

> 6) No resource management nor crash handling for the baloo_file_extractor which
> either OOMs you or corrupts the database on crash leading to 5)

Application crashes cannot corrupt LMDB itself. I can't speak to whatever data 
model Baloo implements on top of that though.

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/

-- 
----------------------------- Dr.-Ing. Christoph Cullmann ---------
AbsInt Angewandte Informatik GmbH      Email: cullmann at AbsInt.com
Science Park 1                         Tel:   +49-681-38360-22
66123 Saarbrücken                      Fax:   +49-681-38360-20
GERMANY                                WWW:   http://www.AbsInt.com
--------------------------------------------------------------------
Geschäftsführung: Dr.-Ing. Christian Ferdinand
Eingetragen im Handelsregister des Amtsgerichts Saarbrücken, HRB 11234