DK Performance: thumbnails and browsing feels extremely sluggish - am I doing anything wrong?

Tue May 9 11:42:04 BST 2023

It's the first time that enabling internal debugging doesn't work for a user. 
Are you possibly using some weird antivirus program that is preventing digiKam 
from setting a simple environment variable?

Maik

Am Dienstag, 9. Mai 2023, 12:32:19 CEST schrieb Maik Qualmann:
> You're doing everything right, it's very strange that no debug messages are
> output. There is a second way to enable debugging. To do this, a variable
> must be set in the Windows environment variable editor. It is described
> here:
> 
> https://www.digikam.org/contribute/
> 
> name: "QT_LOGGING_RULES"
> value: "digikam*=true"
> 
> Maik
> 
> Am Dienstag, 9. Mai 2023, 09:38:55 CEST schrieb Thomas:
> > I just tried again to first uncheck this option. Then close DK.
> > 
> > Then open DK again and then check this option. Then close DK.
> > 
> > Start dbgview.
> > 
> > Start DK again.
> > 
> > 
> > Only the below is logged. No further logs produced when I click around
> > inside DK.
> > 
> > 
> > [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> > [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> > [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> > [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> > [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> > [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> > [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> > [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> > [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> > [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> > [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> > [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> > [26652] kf.config.core: Use of KConfigWatcher without DBus support. You
> > will not receive updates
> > [26652] kf.xmlgui: Unhandled container to remove : Digikam::DigikamApp
> > [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> > [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> > [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> > [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> > [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> > [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> > [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> > [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> > [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> > [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> > [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> > [26652] kf.sonnet.core: No language dictionaries for the language: "en_US"
> > [26652] QLayout: Attempting to add QLayout "" to QWidget "", which
> > already has a layout
> > 
> > On 2023-05-09 09:33, Gilles Caulier wrote:
> > > You don't needs debug symbols version. Debug traces are only prints on
> > > the console. Check if you have enabled the internal debug logging
> > > option from digiKam Setup/Miscs/System dialog page:
> > > 
> > > https://docs.digikam.org/en/setup_application/miscs_settings.html#system
> > > -s
> > > ettings
> > > 
> > > Default settings from DebugView is enough.
> > > 
> > > Best
> > > 
> > > Gilles Caulier
> > > 
> > > Le mar. 9 mai 2023 à 09:25, Thomas <sdktda at gmail.com> a écrit :
> > >> I tried running dbgview as Administrator and enabling various other
> > >> capture options such as kernel capture, global capture etc. However, it
> > >> did not seem to make a difference.
> > >> 
> > >> Do I need a special version of DK with debug symbols compiled in or
> > >> something like that?
> > >> 
> > >> On 2023-05-08 18:48, Maik Qualmann wrote:
> > >>> Create a DebugView log (download from Microsoft). Activate in the
> > >>> digiKam
> > >>> settings under System-> internal debugging, start digiKam again. Start
> > >>> DebugView before, do things in digiKam that are slow and post the
> > >>> contents of the DebugView window.
> > >>> 
> > >>> Maik
> > >>> 
> > >>> Am Montag, 8. Mai 2023, 16:01:34 CEST schrieb Thomas:
> > >>>> Hi,
> > >>>> 
> > >>>> 
> > >>>> First some background:
> > >>>> 
> > >>>> I have a fairly big collection.
> > >>>> 
> > >>>> Currently at more than 700 GB and more than 400k images.
> > >>>> 
> > >>>> Collection is hosted on a NAS over 1 GbE LAN.
> > >>>> 
> > >>>> The NAS server is plenty fast Xeon machine with 4 disks in RAID-1.
> > >>>> The
> > >>>> files are shared via a samba server on Debian.
> > >>>> 
> > >>>> I have various client machines but they all express similar behavior.
> > >>>> The one I use the most is a fairly recent Windows machine with  i7
> > >>>> CPU
> > >>>> and 64 GB RAM and NVMe disk.
> > >>>> 
> > >>>> My collection databases and sizes are shown below. Database files are
> > >>>> hosted locally on the NVMe.
> > >>>> 
> > >>>> 
> > >>>> The behavior I experience:
> > >>>> 
> > >>>> When I click a person in "People" tab, it is often many seconds or
> > >>>> even
> > >>>> minutes before it shows the actual thumbnails of faces for this
> > >>>> person.
> > >>>> 
> > >>>> I just tested this right now by clicking a random person in the list.
> > >>>> There were only 9 images of this person and it took more than 15
> > >>>> seconds
> > >>>> before the thumbnails were shown.
> > >>>> 
> > >>>> I took another person and did the same. This person had more than 8k
> > >>>> images. They were shown immediately in the top. But scrolled a but
> > >>>> down,
> > >>>> the thumbs were blank. So I did that and waited. It took several
> > >>>> minutes
> > >>>> (more than 2) for the thumbs to be shown this far down (probably
> > >>>> about
> > >>>> 10 % scroll down). I then scrolled a bit further down and they were
> > >>>> blank also. Took several minutes for DK to show thumbs.
> > >>>> 
> > >>>> 
> > >>>> Are these thumbnails not cached in the thumbnail database?
> > >>>> 
> > >>>> I mean, all the DK database files are less than 6 GB. They can easily
> > >>>> fit in RAM. Even if they had to be read from the NVMe, the entire 6
> > >>>> GB
> > >>>> can be read from NVMe disk in less than 7 seconds. (tested it using
> > >>>> raw
> > >>>> read of the files without them being cached).
> > >>>> 
> > >>>> 
> > >>>> Another issue happens when I go to Albums and find some image. Then
> > >>>> doubleclick it to open the image in preview mode. It often takes
> > >>>> several
> > >>>> seconds to open the image. Now, I am not sure if the preview is
> > >>>> actually
> > >>>> loaded from the NAS or if it is loaded via the thumbnaildb? But it
> > >>>> not
> > >>>> unusual for this to take 5 seconds or more. This makes browsing
> > >>>> images
> > >>>> feel extremely sluggish.
> > >>>> 
> > >>>> 
> > >>>> So what is happening here? Is it something wrong in my setup?
> > >>>> 
> > >>>> 
> > >>>> What is the most likely bottleneck here?
> > >>>> 
> > >>>>    1. The database files? If so, are they properly indexed? Are the
> > >>>>    proper
> > >>>>    
> > >>>>       settings used relating to sync, locking, etc? Are the databases
> > >>>>       loaded into memory or cached in memory when there is sufficient
> > >>>>       RAM?
> > >>>>    
> > >>>>    2. Is the NAS to blame? I monitor performence metrics relating to
> > >>>>    disk
> > >>>>    
> > >>>>       and I/O on the machine and I see no obvious bottlenecks / high
> > >>>>       utilization on the server while doing the above actions with
> > >>>>       DK.
> > >>>>    
> > >>>>    3. Is the samba network protocol to blame?
> > >>>>    4. Hardware on client or server (I have a hard time seeing this
> > >>>>    being
> > >>>>    
> > >>>>       the case)
> > >>>>    
> > >>>>    5. Is it the network bandwidth between NAS and clients? This is
> > >>>>    low
> > >>>>    
> > >>>>       latency 1 GbE ethernet. It can easily do about 100 MB/s and I
> > >>>>       have
> > >>>>       verified this using iperf.
> > >>>> 
> > >>>> Could it be something else entirely?
> > >>>> 
> > >>>> 
> > >>>> I would love to hear other users' experiences with how DK performs as
> > >>>> well as your collection/db sizes as well as client and server specs.
> > >> 
> > >> --
> > >> Mvh
> > >> Thomas