DK Performance: thumbnails and browsing feels extremely sluggish - am I doing anything wrong?

[Thomas]

Hi,


First some background:

I have a fairly big collection.

Currently at more than 700 GB and more than 400k images.

Collection is hosted on a NAS over 1 GbE LAN.

The NAS server is plenty fast Xeon machine with 4 disks in RAID-1. The 
files are shared via a samba server on Debian.

I have various client machines but they all express similar behavior. 
The one I use the most is a fairly recent Windows machine with  i7 CPU 
and 64 GB RAM and NVMe disk.

My collection databases and sizes are shown below. Database files are 
hosted locally on the NVMe.


The behavior I experience:

When I click a person in "People" tab, it is often many seconds or even 
minutes before it shows the actual thumbnails of faces for this person.

I just tested this right now by clicking a random person in the list. 
There were only 9 images of this person and it took more than 15 seconds 
before the thumbnails were shown.

I took another person and did the same. This person had more than 8k 
images. They were shown immediately in the top. But scrolled a but down, 
the thumbs were blank. So I did that and waited. It took several minutes 
(more than 2) for the thumbs to be shown this far down (probably about 
10 % scroll down). I then scrolled a bit further down and they were 
blank also. Took several minutes for DK to show thumbs.


Are these thumbnails not cached in the thumbnail database?

I mean, all the DK database files are less than 6 GB. They can easily 
fit in RAM. Even if they had to be read from the NVMe, the entire 6 GB 
can be read from NVMe disk in less than 7 seconds. (tested it using raw 
read of the files without them being cached).


Another issue happens when I go to Albums and find some image. Then 
doubleclick it to open the image in preview mode. It often takes several 
seconds to open the image. Now, I am not sure if the preview is actually 
loaded from the NAS or if it is loaded via the thumbnaildb? But it not 
unusual for this to take 5 seconds or more. This makes browsing images 
feel extremely sluggish.


So what is happening here? Is it something wrong in my setup?


What is the most likely bottleneck here?

 1. The database files? If so, are they properly indexed? Are the proper
    settings used relating to sync, locking, etc? Are the databases
    loaded into memory or cached in memory when there is sufficient RAM?
 2. Is the NAS to blame? I monitor performence metrics relating to disk
    and I/O on the machine and I see no obvious bottlenecks / high
    utilization on the server while doing the above actions with DK.
 3. Is the samba network protocol to blame?
 4. Hardware on client or server (I have a hard time seeing this being
    the case)
 5. Is it the network bandwidth between NAS and clients? This is low
    latency 1 GbE ethernet. It can easily do about 100 MB/s and I have
    verified this using iperf.

Could it be something else entirely?


I would love to hear other users' experiences with how DK performs as 
well as your collection/db sizes as well as client and server specs.




-- 
Mvh
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/digikam-users/attachments/20230508/a7634380/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 18axvBqSTdEN1yRf.png
Type: image/png
Size: 11315 bytes
Desc: not available
URL: <http://mail.kde.org/pipermail/digikam-users/attachments/20230508/a7634380/attachment.png>