[KPhotoAlbum] Search performance

[Robert Krawitz]

On Wed, 17 Oct 2018 21:45:07 +0200, Johannes Zarl-Zierl wrote:
> Hi Robert,
>
> Am Mittwoch, 17. Oktober 2018, 03:05:35 CEST schrieb Robert Krawitz:
>> After looking at the code and profiles, it's going to be hard to do a
>> lot better unless we change the internal representation of category
>> items. 
>
> One idea that I was mildly interested in lately was to use an
> in-memory sqlite database or something like that (while keeping the
> index.xml as canonical on- disk format).

This would carry a penalty every time we start kpa.  I'm not sure how
severe the penalty would be, but I'd be very surprised if it were less
than 20% (and possibly considerably more).

I don't remember why we abandoned the SQLDB backend, but if we want an
SQL database, I think it would make more sense to store the data in
one in the first place.  It would certainly make saving a lot faster,
since that would be a database update rather than a full save from
scratch.  The index.xml format is certainly convenient for testing (it
can be edited by hand), but it's not likely the highest performance
way of doing the job.

My thought is that rather than storing collections of category members
as strings that we store them as integer indices into tables of
category members.  The compact index.xml representation already does
that, so we do have some code in kpa that knows how to do this.

My search case was rather extreme, and taking 4 seconds for that isn't
too bad.  But it sure would be nice if even really big searches were
instantaneous.

> I have to admit though, that I have no idea how that would affect
> performance at all. My reasoning was more based on flexibility of
> search…

-- 
Robert Krawitz                                     <rlk at alum.mit.edu>

***  MIT Engineers   A Proud Tradition   http://mitathletics.com  ***
Member of the League for Programming Freedom  --  http://ProgFree.org
Project lead for Gutenprint   --    http://gimp-print.sourceforge.net

"Linux doesn't dictate how I work, I dictate how Linux works."
--Eric Crampton