preview in dolphin is slow with many pictures

[Peter Penz]

Hi Mathias,

On Wednesday, 11. June 2008 22:34:06 Mathias Kraus wrote:
[...]
> > It works very well here with folders having a lot of images (> 1000
> > images), but I agree that turning on previews in /usr/bin is a huge
> > problem (I was not aware about this). I guess this is an issue in kdelibs
> > PreviewJob, where it is timeconsuming to retrieve the MIME-type from the
> > files in usr/bin...
>
> Hmm, maybe I need to delete my build directory and try it again.

Did you also build Qt in the release mode? I remember that shortly before KDE 
4.0 has been released that I was very concerned about the extremely slow 
performance in Dolphin for previews. Although the previews where generated in 
background by another thread, the GUI was not very responsive anymore and the 
general performance was very slow... (I only worked with libraries build in 
the debug mode). It was too late for doing any fixes and 1 week later I 
installed official Suse packages for KDE 4.0 and the performance improved 
dramatically (I'd say up to the factor of 5).

The only difference has been that the Suse packages had been build in the 
release mode. I was quite surprised that in the case of previews there seems 
to be a very, very timeconsuming debugging code in background.

Still also in KDE 4.0 opening /usr/bin having 2300 items with preview takes 
around 4 seconds until the first items appear. But after this 4 seconds the 
GUI is very responsive although Dolphin tries to generate 2300 preview images 
in background.

I did some analyzes yesterday for KDE 4.1. I found a bottleneck in Dolphin 
itself, where the check if an item is visible or not (previews for visible 
items should be generated first) is very time consuming when having 2300 
items like in /usr/bin.

I committed an update and now even in the debug-mode Dolphin is only (?) 
blocked for 2 seconds when opening /usr/bin (before yesterdays patch it was 
blocked up to 4 seconds). Without previews Dolphin requires 1 second to open 
2300 items.

Still using the scrollbar during the preview generation is extremely slow in 
the debug mode, but this will not occur anymore in the release mode. This 
assumption is based on my experience with KDE 4.0 - I have not build Qt, 
kdesupport, kdelibs, kdebase in the release mode yet, but I'm confident that 
the GUI will not get blocked as:
- In this case in KDE 4.1 the same KIO job is started in background like in 
KDE 4.0.
- In KDE 4.1 the KIO job even gets suspended when the scrollbar is touched.

[...]
> > It is already done this way. The previews are generated by a different
> > thread and the user interface is (at least in the case of image previews)
> > not blocked. The creating of previews is paused as soon as the user
> > touches the scrollbars.
>
> I think I really need to delete the build directory and try it again :)

I'm not sure if you have enough time, but I think it would be important also 
to build Qt in the release mode :-)

[...]
> > BTW: can you confirm in your environment that when going into a folder
> > with e. g. 1000 images that no blocking of the GUI occurs?
>
> It still blocks the UI for several seconds, but I'm rebuilding kdelibs and
> kdebase to test if it's the fault of my system.

When discussing the blocking of the UI it is important to differ between 2 
states:
1. The time the UI is blocked when entering the directory.
2. Whether the UI is blocked when using the scrollbar after the directory has 
been loaded.

ad 1: Opening several 1000 items in a directory takes some time when having a 
cold disc cache also without previews. Although the KIO jobs take care giving 
Dolphin feedback about the current percentage of loaded items, the UI still 
might get blocked as layouting and sorting several 1000 items just takes a 
little bit time. Now when creating previews, the following things are done 
internally:

- Check which items are visible and generate the previews for those items 
first. The check is also quite expensive as we need to have a layout already 
to know whether an item is visible.

- Check which MIME-type the items have to know which kind of preview must be 
generated. The check is done in the context of the Dolphin thread and takes 
also a little bit time, but it is less dramatic as I initially thought.

In the debug mode it takes < 1 second on my system to open a directory with 
800 images (preview turned on). It takes another 1 - 2 seconds, until all 
previews for the visible items get generated.

as 2: This is the point I'm more concerned. When the user uses the scrollbars 
or hovers an item, the UI may not be blocked when the previews are generated 
in background. In the debug mode it is blocked: when I use the scrollbar in a 
directory with 2300 items, currently the scrollbar is blocked for 2 (!) 
seconds until everything runs smooth. As mentioned above Dolphin in KDE 4.1 
does exactly the same as Dolphin in KDE 4.0 in this case. In 4.0 (release 
mode) the UI is not blocked at all. Even better: Dolphin in KDE 4.1 even 
pauses the preview jobs when the scrollbar is touched (I checked whether the 
pausing might take so long, but this is not the case). There must be really a 
very expensive debugging code somewhere when creating previews...

> Can anyone else confirm Peter that preview doesn't block the UI?
> If someone needs some images, execute the following script. It takes a
> while, but after that you have 2000+ images

Cool script, thanks :-)

BTW: I also played around just creating previews for the visible area, but 
after checking Konqueror in KDE 3 where the creating of previews in 
background also runs very smooth I threw away the idea again. Beside some 
ongoing optimizations I'd like to keep the current approach for KDE 4.1, as 
we are already in a feature freeze and I don't want to risk any regressions.

Best regards,
Peter

> #!/bin/bash
>
> for((jahr=2000; jahr<=2008; jahr++))
> do
> 	for((monat=1; monat<=9; monat++))
> 	do
> 		for((tag=1; tag<=9; tag++))
> 		do
> 			wget http://www.sinfest.net/comikaze/comics/$jahr-0$monat-0$tag.gif
> 		done
> 		for((tag=10; tag<=31; tag++))
> 		do
> 			wget http://www.sinfest.net/comikaze/comics/$jahr-0$monat-$tag.gif
> 		done
> 	done
> 	for((monat=10; monat<=12; monat++))
> 	do
> 		for((tag=1; tag<=9; tag++))
> 		do
> 			wget http://www.sinfest.net/comikaze/comics/$jahr-$monat-0$tag.gif
> 		done
> 		for((tag=10; tag<=31; tag++))
> 		do
> 			wget http://www.sinfest.net/comikaze/comics/$jahr-$monat-$tag.gif
> 		done
> 	done
> done
>
>
> Thanks, I really appreciate your effort.
>
> > Thanks!
> > Peter
> >
> > > In opposite to Gwenview Dolphin tries to create previews for each
> > > MIME-type.
> >
> >[...]