[KPhotoAlbum] Thumbnail Performance & Compatibility: ImageManager vs KIO

[Johannes Zarl]

Hi Robert,

Thanks for taking the time and giving me some feedback on this.

On Saturday 29 June 2013 18:44:08 Robert Krawitz wrote:
> ++ Performance, performance, and performance.  Oh, and performance while
> we're at it.
> The usual way of storing thumbnails as one image (png or jpg) per source
> file is extremely inefficient.

Ok, I take it that the 10th of a second that other programs need for loading 
the thumbnails is just too slow for you, then ;-)

Seriously, though: I do understand that making KPA slower is just not an 
option. 

> Reading each thumbnail requires a
> context switch into the kernel at least, and if the file isn't in cache,
> at least one I/O operation (it may require reading the directory in
> addition to reading the file from disk).  Rotating media can't sustain
> much more than 100 IOPS, which limits how quickly kpa can display
> thumbnails.  In addition, there's some overhead from decoding the image
> files.

Maybe I'm wrong, but I thought that's also true of the current way things are 
done in KPA. After all, the thumbs-X file is opened and closed for every 
thumbnail that is being loaded. Context-switch-wise this should be about the 
same overhead as with multiple single thumbnails. And I'm also not so sure 
about file-system caching doing any good here either, since the current scheme 
performs extremely poor on network-mounts (Though probably not worse than a 
single-thumbnail-per-file scheme when the thumbnails directory is on the 
network share)

> (It also significantly reduces the storage required -- even though the
> thumbnails are stored decoded, they take up fewer disk blocks, because
> they avoid filesystem overhead for small files.)

Actually, it roughly doubles the storage required. After all you have one copy 
of the slightly less efficiently stored thumbnails, used by almost all other 
applications, plus one copy of the slightly more efficiently stored 
thumbnails, used by KPA.

If we already store the thumbnails twice, we could at least generate them only 
once. By using PreviewJob to create the thumbnail that we put into out 
thumbnailcache, we would effectively half the time spent on thumbnail 
generation.

> [...] I think the tradeoff Jesper made is a good one.

I'll take that as a "good job" to Jesper and a "don't do anything stupid" to 
me ;-)

Cheers,
  Johannes