[KPhotoAlbum] Search performance

[Joe]

On 10/17/2018 08:37 PM, Robert Krawitz wrote:
> On Wed, 17 Oct 2018 21:45:07 +0200, Johannes Zarl-Zierl wrote:
>> Hi Robert,
>>
>> Am Mittwoch, 17. Oktober 2018, 03:05:35 CEST schrieb Robert Krawitz:
>>> After looking at the code and profiles, it's going to be hard to do a
>>> lot better unless we change the internal representation of category
>>> items. 
>> One idea that I was mildly interested in lately was to use an
>> in-memory sqlite database or something like that (while keeping the
>> index.xml as canonical on- disk format).
> This would carry a penalty every time we start kpa.  I'm not sure how
> severe the penalty would be, but I'd be very surprised if it were less
> than 20% (and possibly considerably more).
>
> I don't remember why we abandoned the SQLDB backend, but if we want an
> SQL database, I think it would make more sense to store the data in
> one in the first place.  It would certainly make saving a lot faster,
> since that would be a database update rather than a full save from
> scratch.  The index.xml format is certainly convenient for testing (it
> can be edited by hand), but it's not likely the highest performance
> way of doing the job
>
> My thought is that rather than storing collections of category members
> as strings that we store them as integer indices into tables of
> category members.  The compact index.xml representation already does
> that, so we do have some code in kpa that knows how to do this.
>
> My search case was rather extreme, and taking 4 seconds for that isn't
> too bad.  But it sure would be nice if even really big searches were
> instantaneous.
>
>> I have to admit though, that I have no idea how that would affect
>> performance at all. My reasoning was more based on flexibility of
>> search…
I believe I saw comments on this list that the SQL code was very broken
and that was the reason for abandoning it at the time.

All this performance work/internals is stuff an end user would take for
granted (no glory), but it really helps. Thanks for all the hard work.

Joe