DK Performance: thumbnails and browsing feels extremely sluggish - am I doing anything wrong?

[Thomas]

I will try that.

Does the generated information contain any PID data or other sensitive 
data such as hostnames, file or folder names of any of my photos or 
albums, etc?




On 2023-05-08 18:48, Maik Qualmann wrote:
> Create a DebugView log (download from Microsoft). Activate in the digiKam
> settings under System-> internal debugging, start digiKam again. Start
> DebugView before, do things in digiKam that are slow and post the contents of
> the DebugView window.
>
> Maik
>
> Am Montag, 8. Mai 2023, 16:01:34 CEST schrieb Thomas:
>> Hi,
>>
>>
>> First some background:
>>
>> I have a fairly big collection.
>>
>> Currently at more than 700 GB and more than 400k images.
>>
>> Collection is hosted on a NAS over 1 GbE LAN.
>>
>> The NAS server is plenty fast Xeon machine with 4 disks in RAID-1. The
>> files are shared via a samba server on Debian.
>>
>> I have various client machines but they all express similar behavior.
>> The one I use the most is a fairly recent Windows machine with  i7 CPU
>> and 64 GB RAM and NVMe disk.
>>
>> My collection databases and sizes are shown below. Database files are
>> hosted locally on the NVMe.
>>
>>
>> The behavior I experience:
>>
>> When I click a person in "People" tab, it is often many seconds or even
>> minutes before it shows the actual thumbnails of faces for this person.
>>
>> I just tested this right now by clicking a random person in the list.
>> There were only 9 images of this person and it took more than 15 seconds
>> before the thumbnails were shown.
>>
>> I took another person and did the same. This person had more than 8k
>> images. They were shown immediately in the top. But scrolled a but down,
>> the thumbs were blank. So I did that and waited. It took several minutes
>> (more than 2) for the thumbs to be shown this far down (probably about
>> 10 % scroll down). I then scrolled a bit further down and they were
>> blank also. Took several minutes for DK to show thumbs.
>>
>>
>> Are these thumbnails not cached in the thumbnail database?
>>
>> I mean, all the DK database files are less than 6 GB. They can easily
>> fit in RAM. Even if they had to be read from the NVMe, the entire 6 GB
>> can be read from NVMe disk in less than 7 seconds. (tested it using raw
>> read of the files without them being cached).
>>
>>
>> Another issue happens when I go to Albums and find some image. Then
>> doubleclick it to open the image in preview mode. It often takes several
>> seconds to open the image. Now, I am not sure if the preview is actually
>> loaded from the NAS or if it is loaded via the thumbnaildb? But it not
>> unusual for this to take 5 seconds or more. This makes browsing images
>> feel extremely sluggish.
>>
>>
>> So what is happening here? Is it something wrong in my setup?
>>
>>
>> What is the most likely bottleneck here?
>>
>>   1. The database files? If so, are they properly indexed? Are the proper
>>      settings used relating to sync, locking, etc? Are the databases
>>      loaded into memory or cached in memory when there is sufficient RAM?
>>   2. Is the NAS to blame? I monitor performence metrics relating to disk
>>      and I/O on the machine and I see no obvious bottlenecks / high
>>      utilization on the server while doing the above actions with DK.
>>   3. Is the samba network protocol to blame?
>>   4. Hardware on client or server (I have a hard time seeing this being
>>      the case)
>>   5. Is it the network bandwidth between NAS and clients? This is low
>>      latency 1 GbE ethernet. It can easily do about 100 MB/s and I have
>>      verified this using iperf.
>>
>> Could it be something else entirely?
>>
>>
>> I would love to hear other users' experiences with how DK performs as
>> well as your collection/db sizes as well as client and server specs.
>
>
>
-- 
Mvh
Thomas