DK Performance: thumbnails and browsing feels extremely sluggish - am I doing anything wrong?

[Maik Qualmann]

Create a DebugView log (download from Microsoft). Activate in the digiKam 
settings under System-> internal debugging, start digiKam again. Start 
DebugView before, do things in digiKam that are slow and post the contents of 
the DebugView window.

Maik

Am Montag, 8. Mai 2023, 16:01:34 CEST schrieb Thomas:
> Hi,
> 
> 
> First some background:
> 
> I have a fairly big collection.
> 
> Currently at more than 700 GB and more than 400k images.
> 
> Collection is hosted on a NAS over 1 GbE LAN.
> 
> The NAS server is plenty fast Xeon machine with 4 disks in RAID-1. The
> files are shared via a samba server on Debian.
> 
> I have various client machines but they all express similar behavior.
> The one I use the most is a fairly recent Windows machine with  i7 CPU
> and 64 GB RAM and NVMe disk.
> 
> My collection databases and sizes are shown below. Database files are
> hosted locally on the NVMe.
> 
> 
> The behavior I experience:
> 
> When I click a person in "People" tab, it is often many seconds or even
> minutes before it shows the actual thumbnails of faces for this person.
> 
> I just tested this right now by clicking a random person in the list.
> There were only 9 images of this person and it took more than 15 seconds
> before the thumbnails were shown.
> 
> I took another person and did the same. This person had more than 8k
> images. They were shown immediately in the top. But scrolled a but down,
> the thumbs were blank. So I did that and waited. It took several minutes
> (more than 2) for the thumbs to be shown this far down (probably about
> 10 % scroll down). I then scrolled a bit further down and they were
> blank also. Took several minutes for DK to show thumbs.
> 
> 
> Are these thumbnails not cached in the thumbnail database?
> 
> I mean, all the DK database files are less than 6 GB. They can easily
> fit in RAM. Even if they had to be read from the NVMe, the entire 6 GB
> can be read from NVMe disk in less than 7 seconds. (tested it using raw
> read of the files without them being cached).
> 
> 
> Another issue happens when I go to Albums and find some image. Then
> doubleclick it to open the image in preview mode. It often takes several
> seconds to open the image. Now, I am not sure if the preview is actually
> loaded from the NAS or if it is loaded via the thumbnaildb? But it not
> unusual for this to take 5 seconds or more. This makes browsing images
> feel extremely sluggish.
> 
> 
> So what is happening here? Is it something wrong in my setup?
> 
> 
> What is the most likely bottleneck here?
> 
>  1. The database files? If so, are they properly indexed? Are the proper
>     settings used relating to sync, locking, etc? Are the databases
>     loaded into memory or cached in memory when there is sufficient RAM?
>  2. Is the NAS to blame? I monitor performence metrics relating to disk
>     and I/O on the machine and I see no obvious bottlenecks / high
>     utilization on the server while doing the above actions with DK.
>  3. Is the samba network protocol to blame?
>  4. Hardware on client or server (I have a hard time seeing this being
>     the case)
>  5. Is it the network bandwidth between NAS and clients? This is low
>     latency 1 GbE ethernet. It can easily do about 100 MB/s and I have
>     verified this using iperf.
> 
> Could it be something else entirely?
> 
> 
> I would love to hear other users' experiences with how DK performs as
> well as your collection/db sizes as well as client and server specs.