DK Performance: thumbnails and browsing feels extremely sluggish - am I doing anything wrong?

Tue May 9 11:32:19 BST 2023

You're doing everything right, it's very strange that no debug messages are 
output. There is a second way to enable debugging. To do this, a variable must 
be set in the Windows environment variable editor. It is described here:

https://www.digikam.org/contribute/

name: "QT_LOGGING_RULES"
value: "digikam*=true"

Maik

Am Dienstag, 9. Mai 2023, 09:38:55 CEST schrieb Thomas:
> I just tried again to first uncheck this option. Then close DK.
> 
> Then open DK again and then check this option. Then close DK.
> 
> Start dbgview.
> 
> Start DK again.
> 
> 
> Only the below is logged. No further logs produced when I click around
> inside DK.
> 
> 
> [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> [26652] kf.config.core: Use of KConfigWatcher without DBus support. You
> will not receive updates
> [26652] kf.xmlgui: Unhandled container to remove : Digikam::DigikamApp
> [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> [26652] QLayout: Attempting to add QLayout "" to QWidget "", which
> already has a layout
> 
> On 2023-05-09 09:33, Gilles Caulier wrote:
> > You don't needs debug symbols version. Debug traces are only prints on
> > the console. Check if you have enabled the internal debug logging
> > option from digiKam Setup/Miscs/System dialog page:
> > 
> > https://docs.digikam.org/en/setup_application/miscs_settings.html#system-s
> > ettings
> > 
> > Default settings from DebugView is enough.
> > 
> > Best
> > 
> > Gilles Caulier
> > 
> > Le mar. 9 mai 2023 à 09:25, Thomas <sdktda at gmail.com> a écrit :
> >> I tried running dbgview as Administrator and enabling various other
> >> capture options such as kernel capture, global capture etc. However, it
> >> did not seem to make a difference.
> >> 
> >> Do I need a special version of DK with debug symbols compiled in or
> >> something like that?
> >> 
> >> On 2023-05-08 18:48, Maik Qualmann wrote:
> >>> Create a DebugView log (download from Microsoft). Activate in the
> >>> digiKam
> >>> settings under System-> internal debugging, start digiKam again. Start
> >>> DebugView before, do things in digiKam that are slow and post the
> >>> contents of the DebugView window.
> >>> 
> >>> Maik
> >>> 
> >>> Am Montag, 8. Mai 2023, 16:01:34 CEST schrieb Thomas:
> >>>> Hi,
> >>>> 
> >>>> 
> >>>> First some background:
> >>>> 
> >>>> I have a fairly big collection.
> >>>> 
> >>>> Currently at more than 700 GB and more than 400k images.
> >>>> 
> >>>> Collection is hosted on a NAS over 1 GbE LAN.
> >>>> 
> >>>> The NAS server is plenty fast Xeon machine with 4 disks in RAID-1. The
> >>>> files are shared via a samba server on Debian.
> >>>> 
> >>>> I have various client machines but they all express similar behavior.
> >>>> The one I use the most is a fairly recent Windows machine with  i7 CPU
> >>>> and 64 GB RAM and NVMe disk.
> >>>> 
> >>>> My collection databases and sizes are shown below. Database files are
> >>>> hosted locally on the NVMe.
> >>>> 
> >>>> 
> >>>> The behavior I experience:
> >>>> 
> >>>> When I click a person in "People" tab, it is often many seconds or even
> >>>> minutes before it shows the actual thumbnails of faces for this person.
> >>>> 
> >>>> I just tested this right now by clicking a random person in the list.
> >>>> There were only 9 images of this person and it took more than 15
> >>>> seconds
> >>>> before the thumbnails were shown.
> >>>> 
> >>>> I took another person and did the same. This person had more than 8k
> >>>> images. They were shown immediately in the top. But scrolled a but
> >>>> down,
> >>>> the thumbs were blank. So I did that and waited. It took several
> >>>> minutes
> >>>> (more than 2) for the thumbs to be shown this far down (probably about
> >>>> 10 % scroll down). I then scrolled a bit further down and they were
> >>>> blank also. Took several minutes for DK to show thumbs.
> >>>> 
> >>>> 
> >>>> Are these thumbnails not cached in the thumbnail database?
> >>>> 
> >>>> I mean, all the DK database files are less than 6 GB. They can easily
> >>>> fit in RAM. Even if they had to be read from the NVMe, the entire 6 GB
> >>>> can be read from NVMe disk in less than 7 seconds. (tested it using raw
> >>>> read of the files without them being cached).
> >>>> 
> >>>> 
> >>>> Another issue happens when I go to Albums and find some image. Then
> >>>> doubleclick it to open the image in preview mode. It often takes
> >>>> several
> >>>> seconds to open the image. Now, I am not sure if the preview is
> >>>> actually
> >>>> loaded from the NAS or if it is loaded via the thumbnaildb? But it not
> >>>> unusual for this to take 5 seconds or more. This makes browsing images
> >>>> feel extremely sluggish.
> >>>> 
> >>>> 
> >>>> So what is happening here? Is it something wrong in my setup?
> >>>> 
> >>>> 
> >>>> What is the most likely bottleneck here?
> >>>> 
> >>>>    1. The database files? If so, are they properly indexed? Are the
> >>>>    proper
> >>>>    
> >>>>       settings used relating to sync, locking, etc? Are the databases
> >>>>       loaded into memory or cached in memory when there is sufficient
> >>>>       RAM?
> >>>>    
> >>>>    2. Is the NAS to blame? I monitor performence metrics relating to
> >>>>    disk
> >>>>    
> >>>>       and I/O on the machine and I see no obvious bottlenecks / high
> >>>>       utilization on the server while doing the above actions with DK.
> >>>>    
> >>>>    3. Is the samba network protocol to blame?
> >>>>    4. Hardware on client or server (I have a hard time seeing this
> >>>>    being
> >>>>    
> >>>>       the case)
> >>>>    
> >>>>    5. Is it the network bandwidth between NAS and clients? This is low
> >>>>    
> >>>>       latency 1 GbE ethernet. It can easily do about 100 MB/s and I
> >>>>       have
> >>>>       verified this using iperf.
> >>>> 
> >>>> Could it be something else entirely?
> >>>> 
> >>>> 
> >>>> I would love to hear other users' experiences with how DK performs as
> >>>> well as your collection/db sizes as well as client and server specs.
> >> 
> >> --
> >> Mvh
> >> Thomas