[KPhotoAlbum] Search performance

[Robert Krawitz]

On Thu, 18 Oct 2018 07:31:22 +0200, Martin Hoeller wrote:
> On 17 Okt 2018, Robert Krawitz wrote:
>> On Wed, 17 Oct 2018 21:45:07 +0200, Johannes Zarl-Zierl wrote:
>> > Hi Robert,
>> >
>> > Am Mittwoch, 17. Oktober 2018, 03:05:35 CEST schrieb Robert Krawitz:  
>> >> After looking at the code and profiles, it's going to be hard to do a
>> >> lot better unless we change the internal representation of category
>> >> items.   
>> >
>> > One idea that I was mildly interested in lately was to use an
>> > in-memory sqlite database or something like that (while keeping the
>> > index.xml as canonical on- disk format).  
> [...]
>> I don't remember why we abandoned the SQLDB backend, 
>
> The reason seemed to be complexity. Read this email thread from 6 years
> ago: https://mail.kdab.com/pipermail/kphotoalbum/2012-April/005024.html

Yes, I can well understand why.

>> but if we want an
>> SQL database, I think it would make more sense to store the data in
>> one in the first place.
>
> Well, I like the idea of an SQL backend... the question is: can we manage
> this task. As far as I know we have 3 active committers (all with limited
> time). So as already 3 attempts were made an failed, I doubt that another
> attempty would succeed.

It certainly would not be an easy task; I just think that without it
we're going to have a lot of trouble improving performance very much.

We're spending so much time in string functions, though, that I wonder
whether shortening the XML tag and attribute names (to one
character/byte, perhaps?) would significantly improve load/save
times.  It shrinks the size of my index.xml considerably, from 51 MB
to 40 MB.

We could also consider putting immutable data (in particular, MD5
checksum, image dimensions) in the EXIF database.

All of this has the feel of nibbling at the edges, to be sure.  But I
do note that ODF is quite ruthless about using very short
tag/attribute names.
-- 
Robert Krawitz                                     <rlk at alum.mit.edu>

***  MIT Engineers   A Proud Tradition   http://mitathletics.com  ***
Member of the League for Programming Freedom  --  http://ProgFree.org
Project lead for Gutenprint   --    http://gimp-print.sourceforge.net

"Linux doesn't dictate how I work, I dictate how Linux works."
--Eric Crampton