[KPhotoAlbum] More performance work

[Robert Krawitz]

On Thu, 10 May 2018 22:47:13 -0400 (EDT), Robert Krawitz wrote:
> I know I've been a bit single minded about the load performance, but
> that's really the very first thing people see when they look at kpa or
> any similar program.  I know that when I've played with Digikam the
> image load is so slow that it makes even experimenting painful.
> Loading 3000 images in something under 3 minutes is fast enough that
> even quite large image collections can be tested out without too much
> discomfort.  I'd honestly like to be able to load such small images (1
> MB) even faster than the 16-18 images/second I'm getting, but that
> will require a closer look at what's going on.

Thought something was wrong there.  It was actually off by about an
order of magnitude or so; when I batched up the inserts it took more
like 15 seconds to load the 3000 images (about 200/second).  The
mistake was inserting them one by one into the image info list, which
requires merging them (to preserve sorted order).  Batching them up,
as is currently the case with images not to be stacked, is a lot
cheaper.

It should also make throwing away an aborted load easier, although the
logic's probably still not quite right (it's not inserting them into
the list, but it is inserting them into the hash).  The two ways
around that would be to throw away the hash and rebuild it when needed
on abort, or to keep a pending hash as well as pending unsorted list.

In any event, batching it up still results in loading these images at
about 200 MB/sec; if they weren't all in RAM, I'd be I/O limited by my
spinning rust.  That's what we want when large SSD's become cheap.
-- 
Robert Krawitz                                     <rlk at alum.mit.edu>

***  MIT Engineers   A Proud Tradition   http://mitathletics.com  ***
Member of the League for Programming Freedom  --  http://ProgFree.org
Project lead for Gutenprint   --    http://gimp-print.sourceforge.net

"Linux doesn't dictate how I work, I dictate how Linux works."
--Eric Crampton