Taking over maintainership of Baloo

[Christoph Cullmann]

Hi,

after looking a bit more at the code, I think there are ATM a lot of things that need fixing:

1) 32-bit system: I see no fix, > 1GB of index and baloo + all baloo using applications fail

  see bugs like https://bugs.kde.org/show_bug.cgi?id=356114 here we have the 5GB limit, which is now raised
  for 64-bit, but not for 32-bit

2) Larger filesystems: unfortunately one decided to ignore the upper 32-bit of the inodes

/**
 * Convert the QT_STATBUF into a 64 bit unique identifier for the file.
 * This identifier is combination of the device id and inode number.
 */
inline quint64 statBufToId(const QT_STATBUF& stBuf)
{
    // We're loosing 32 bits of info, so this could potentially break
    // on file systems with really large inode and device ids
    return devIdAndInodeToId(static_cast<quint32>(stBuf.st_dev),
                             static_cast<quint32>(stBuf.st_ino));
}

=> random breakage e.g. on my NFS drive here as the IDs clash and all invariants no longer hold.
(e.g. something can be a file but in addition a directory, ....)

3) No error handling of most lmdb faults (like already mentioned)

4) No error handling for any data corruption: e.g. many places will just endless loop or malloc, like
  DocumentUrlDB::get(quint64 docId) (we have bugs for that)

5) lmdb locking issues: crash one read-write process => all other things stall (or crash because of 3+4)

6) No resource management nor crash handling for the baloo_file_extractor which either OOMs you or corrupts the database on crash leading to 5)

CC'd Vishesh, perhaps I am wrong with that issues and misunderstand the code, unfortunately e.g. the database
structure is not that well documented, if I don't just not find the correct docs in the git.

Greetings
Christoph

----- Am 12. Sep 2016 um 0:54 schrieb cullmann cullmann at absint.com:

> Hi,
> 
>> Hey Boudhayan,
>> 
>> It's nice that you are stepping up for maintaining baloo. I have no
>> objections to that and will try to help you out whenever needed :)
> I triaged the bugs a lot today, we are now down to < 70:
> 
> https://bugs.kde.org/buglist.cgi?bug_severity=critical&bug_severity=grave&bug_severity=major&bug_severity=crash&bug_severity=normal&bug_severity=minor&bug_status=UNCONFIRMED&bug_status=CONFIRMED&bug_status=ASSIGNED&bug_status=REOPENED&list_id=1385670&product=frameworks-baloo
> 
> It would be really great, if you could take a look at some and perhaps
> help with adding the needed error checks.
> 
> Greetings
> Christoph
> 
> --
> ----------------------------- Dr.-Ing. Christoph Cullmann ---------
> AbsInt Angewandte Informatik GmbH      Email: cullmann at AbsInt.com
> Science Park 1                         Tel:   +49-681-38360-22
> 66123 Saarbrücken                      Fax:   +49-681-38360-20
> GERMANY                                WWW:   http://www.AbsInt.com
> --------------------------------------------------------------------
> Geschäftsführung: Dr.-Ing. Christian Ferdinand
> Eingetragen im Handelsregister des Amtsgerichts Saarbrücken, HRB 11234

-- 
----------------------------- Dr.-Ing. Christoph Cullmann ---------
AbsInt Angewandte Informatik GmbH      Email: cullmann at AbsInt.com
Science Park 1                         Tel:   +49-681-38360-22
66123 Saarbrücken                      Fax:   +49-681-38360-20
GERMANY                                WWW:   http://www.AbsInt.com
--------------------------------------------------------------------
Geschäftsführung: Dr.-Ing. Christian Ferdinand
Eingetragen im Handelsregister des Amtsgerichts Saarbrücken, HRB 11234