DK Performance: thumbnails and browsing feels extremely sluggish - am I doing anything wrong?

[Thomas]

FYI: It seems the check in internal logging in DK Settings does not have 
any effect.

When I uncheck this, DK keeps loggigng just as much in dbgview, even 
after restarting DK.

It seems the environment variables take precedence over this setting.





On 2023-05-08 18:48, Maik Qualmann wrote:
> Create a DebugView log (download from Microsoft). Activate in the digiKam
> settings under System-> internal debugging, start digiKam again. Start
> DebugView before, do things in digiKam that are slow and post the contents of
> the DebugView window.
>
> Maik
>
> Am Montag, 8. Mai 2023, 16:01:34 CEST schrieb Thomas:
>> Hi,
>>
>>
>> First some background:
>>
>> I have a fairly big collection.
>>
>> Currently at more than 700 GB and more than 400k images.
>>
>> Collection is hosted on a NAS over 1 GbE LAN.
>>
>> The NAS server is plenty fast Xeon machine with 4 disks in RAID-1. The
>> files are shared via a samba server on Debian.
>>
>> I have various client machines but they all express similar behavior.
>> The one I use the most is a fairly recent Windows machine with  i7 CPU
>> and 64 GB RAM and NVMe disk.
>>
>> My collection databases and sizes are shown below. Database files are
>> hosted locally on the NVMe.
>>
>>
>> The behavior I experience:
>>
>> When I click a person in "People" tab, it is often many seconds or even
>> minutes before it shows the actual thumbnails of faces for this person.
>>
>> I just tested this right now by clicking a random person in the list.
>> There were only 9 images of this person and it took more than 15 seconds
>> before the thumbnails were shown.
>>
>> I took another person and did the same. This person had more than 8k
>> images. They were shown immediately in the top. But scrolled a but down,
>> the thumbs were blank. So I did that and waited. It took several minutes
>> (more than 2) for the thumbs to be shown this far down (probably about
>> 10 % scroll down). I then scrolled a bit further down and they were
>> blank also. Took several minutes for DK to show thumbs.
>>
>>
>> Are these thumbnails not cached in the thumbnail database?
>>
>> I mean, all the DK database files are less than 6 GB. They can easily
>> fit in RAM. Even if they had to be read from the NVMe, the entire 6 GB
>> can be read from NVMe disk in less than 7 seconds. (tested it using raw
>> read of the files without them being cached).
>>
>>
>> Another issue happens when I go to Albums and find some image. Then
>> doubleclick it to open the image in preview mode. It often takes several
>> seconds to open the image. Now, I am not sure if the preview is actually
>> loaded from the NAS or if it is loaded via the thumbnaildb? But it not
>> unusual for this to take 5 seconds or more. This makes browsing images
>> feel extremely sluggish.
>>
>>
>> So what is happening here? Is it something wrong in my setup?
>>
>>
>> What is the most likely bottleneck here?
>>
>>   1. The database files? If so, are they properly indexed? Are the proper
>>      settings used relating to sync, locking, etc? Are the databases
>>      loaded into memory or cached in memory when there is sufficient RAM?
>>   2. Is the NAS to blame? I monitor performence metrics relating to disk
>>      and I/O on the machine and I see no obvious bottlenecks / high
>>      utilization on the server while doing the above actions with DK.
>>   3. Is the samba network protocol to blame?
>>   4. Hardware on client or server (I have a hard time seeing this being
>>      the case)
>>   5. Is it the network bandwidth between NAS and clients? This is low
>>      latency 1 GbE ethernet. It can easily do about 100 MB/s and I have
>>      verified this using iperf.
>>
>> Could it be something else entirely?
>>
>>
>> I would love to hear other users' experiences with how DK performs as
>> well as your collection/db sizes as well as client and server specs.
>
>
>
-- 
Mvh
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/digikam-users/attachments/20230510/32a8f305/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cx04Wj24NcgyQrNV.png
Type: image/png
Size: 18246 bytes
Desc: not available
URL: <http://mail.kde.org/pipermail/digikam-users/attachments/20230510/32a8f305/attachment-0001.png>