[KPhotoAlbum] image decoding priorities & multithreading

[Henner Zeller]

Hi,
On Mon, Jul 14, 2008 at 12:07 PM, Jan Kundrát <jkt at gentoo.org> wrote:
> KPA now does thumbnail preloading -- as soon as the ThumbnailWidget knows
> what images it contains, all of them are decoded in the background. This
> process continues until another bunch of images is to be shown.

Thanks Jan, this sounds like a huge step forward (haven't looked at
the code yet, so maybe I am going to suggest stuff down there what you
already implemented).

>
> This behavior is open for discussion, perhaps it should stop when
> ThumbnailWidget goes out of user's sight, perhaps it should only load images
> that aren't too far from current viewport etc. I'm a bit worried about
> needlessly generating thousands of thumbnails when user opts to view all
> images in her collection... Suggestions?

Personally, I would continue creating thumbnails unless there is
something more important to do. I always find it annoying when I
scroll that I have to wait for the thumbnails to be processed (after I
uploaded another 500 of them). Note, with 'processing' I mean
downscaling the thumbnails from the bigger images because that takes
time, not loading the preprocessed thumbnails. Reason: we need to do
it anyway (unless we change the resolution in the configuration) and
the processor time is better spend ofline when it otherwise would
idle. The thumbnails to be processed should go in both directions of
the current viewport, but with emphasis on the current scolling
direction (something along the lines of: if I scroll down, process
three images 'down' and one image 'up'; when I switch the scrolling
direction, the other way around).

Having said that, the important question is, how to prioritize image
loading ? Next to the obvious (show the thumbnails on the thumbnail
view) it depends on the expected actions by the user: we should
anticipate that and have everything ready before the user chooses that
action. The actions are: showing a picture in 'big'
(window/fullscreen) and scrolling:
So the priorities can be something like this (first pass, I might have
missed something):
   1)  Loading existing thumbnails (that are already available in
~/.thumbnails) for the current viewport. Low hanging fruit and fast.
   2)  Downscaling thumbnails for the current viewport.
   3)  Preload the 'next' fullsize image. If you go on an image with
the cursor and press 'return', the image should already be preloaded.
What do I mean with 'next' ? There should be several preloaded images:
first the current cursor image, then all four surrounding images. What
image to preload first of the surrounding images can be determined by
the last cursor movement (going 'right' or 'left' in the list ?). That
will give instant image popup in almost all cases - a huge impact on
the perceived performance! (Right now, I wait at least 1.5-2 seconds
on my notebook which feels very slow).
   4) loading the next two rows of existing thumbnails up and down the
current viewport to accomodate slow scrolling.
   5) Processing (downscaling) all thumbnails starting from the
current viewport with emphasis on the thumbnails down the current
scrolling direction. Maybe with low operating system priority to not
hog other processes on the machine.

This of course means, that you typically need to reorganize/throw away
pending requests in the queue with every viewport move, but that
effort is worth it. It as well means that you might use CPU time that
you actually don't use later (preloaded big images over the cursor
come to mind). But using otherwise idle CPU time is better than
letting the user wait later. Idle CPU time is lost forever, so better
use it while it is there.

So I disagree that we should stop processing images after some x times
viewport size threshold - consider the user might want to decide to
quickly scroll over the images and then finds out that only 200 of the
5000 images are processed. Especially computationally expensive
downscaling of thumbnails is not lost work (unless we decide to change
the default size of the thumbnails in which case all thumbnails have
to be reprocessed) as they can be loaded from the ~/.thumbnails later
much faster.
Right now the user only can choose the much more awkward 'Regenerate
Thumbnails' to do what KPA should do anyway automatically.

Since I feel strongly about these things, I am happy to work on part
of it if you want.

-henner