ksycoca and kbuildsycoca, the next step

[David Faure]

On Sunday 06 September 2015 18:15:47 Matthew Dawson wrote:
> On September 7, 2015 12:03:59 AM you wrote:
> > On Sunday 06 September 2015 17:59:13 Matthew Dawson wrote:
> > > On September 6, 2015 01:52:37 PM David Faure wrote:
> > > > As a followup to recent discussions, I'm now looking into moving the
> > > > recreating kbuildsycoca directly into the kservice library.
> > > > 
> > > > The rebuild now happens on demand when using any KService API and one
> > > > the
> > > > dirs with desktop files is more recent than the sycoca cache (that part
> > > > is
> > > 
> > > > already in), so the next steps are:
> > > Everything looks good from me, just one comment:
> > > > 1) kded no longer needs to watch the dirs with desktop files.
> > > > I assume even the K menu doesn't just keep a cache of everything in
> > > > memory,
> > > > and use the KService/KServiceGroup API every time it's opened, so it
> > > > should
> > > > get the new stuff. However on demand means the old DBus signal
> > > > databaseChanged() is either killed or at rebuild time by the app doing
> > > > the
> > > > rebuild (which could be later than when using file watching; possibly a
> > > > very long time if no app calls into KService for a long time)...
> > > 
> > > Instead of killing the kded component completely, could we instead make it
> > > optional?  That way in a full KDE session, we monitor the directories as
> > > needed so we don't loose the file watching.  Then if an application using
> > > KService is launched outside the session, it falls back to this behaviour.
> > > 
> > > Though the problem from point 4 still exists.  Maybe make the signal be
> > > specific to the cache?  Since that would be more work then just a straight
> > > port that no one has time to write, I wouldn't care if the kded component
> > > is killed off currently, as long as it isn't a problem to bring back in a
> > > future version where it would be useful.

I had missed the suggestion of "make the signal specific to the cache".
Not sure how this helps; at most it prevents some unnecessary calls to some
slots (which will think a cache got updated when it wasn't), no?

> > I'm missing one point in your reasoning: what would we gain from keeping
> > the kded code that watches desktop files and runs kbuildsycoca,
> > if the apps do the same anyway, any time they need the sycoca contents?
>
> I was thinking for things like the K menu, having a significant pause when 
> opening the menu would be a bad UX.  

Oh, that's a good point indeed. Thanks!

In addition to that, I had a look, and there are indeed many cases where
we need the notification signal.

http://lxr.kde.org/source/frameworks/plasma-framework/src/plasma/private/associatedapplicationmanager.cpp
updates text and icon in some applets representing apps, when sycoca is changed.

These bits of code do similar things.
http://lxr.kde.org/source/kde/kdeplasma-addons/applets/icontasks/mediabuttons.cpp
http://lxr.kde.org/source/kde/kdeplasma-addons/dataengines/comic/comic.cpp
http://lxr.kde.org/source/kde/kdeplasma-addons/libs/lancelot-datamodels/Applications.cpp
They all update internal data structures based on a sycoca query that is made on startup
and when sycoca changes.

There are more hits for databaseChanged, all in plasma-desktop and plasma-workspace.

> And having each app implement its own caching layer would make things worse.

That's definitely not what I meant. I meant app caches are bad, not good, in this case ;)
E.g. what plasma/private/associatedapplicationmanager.cpp does is a kind of cache,
it doesn't query sycoca when needed, it updates when the signal comes in. That's
indeed the only way to do it for things that are always visible like plasma applets
(unlike the K menu). So yes, we need file watching and the signal, at least in a full
plasma session.

All the use cases I found above are in plasma, which makes me wonder if we could do
that file watching in plasma-desktop instead of kded.

Of course I could make up a theoretical case of an application which has always-visible buttons
for starting some file in other apps (associated with the mimetype of the file), and
which would need to update these buttons when sycoca changes, i.e. same needs
as plasma. If we want this to work, we need to keep file watching in all cases. I'm not fond
of that reasoning. Alternative solution: the code for file watching moves to the kservice library
too, and it's called by kded (or plasma, if we decide to move it there) and by that theoretical
application which needs such an immediate reaction to changes in installed applications
or in mimeapps associations by the user.
 
> Also, since most people don't use make install to install/update programs, 
> could we potentially miss something if we only watch the folder's mtime?

I was less worried about distro packages. For sure installing a new program will
touch the mtime of the dir, by definition. Updating a program, however, I'm not sure.
Let's find out.

$ rpm -qil kdelibs4-doc | grep /usr/share/man
/usr/share/man/man1/meinproc4.1.gz
$ stat /usr/share/man/man1 | grep Modify
Modify: 2015-08-04 13:51:08.961542417 +0200
$ sudo zypper in kdelibs4-doc
stat /usr/share/man/man1 | grep Modify
Modify: 2015-09-07 10:17:53.344141470 +0200

Seems to work fine. Quite logical in fact, I suspect all binary package systems
first remove the old package and then install the new one, rather than modifying
existing file in place. Therefore the mtime of the dir is updated, no problem.

==

OK, in summary: we keep file watching in kded or plasma, the big difference with before
is that this isn't triggered automatically by any random app just using sycoca.

At the moment it's still triggered by any use of kded module though, so it should either
move to a kded module (which would be marked as "load on desktop startup"), or to
plasma itself. (Any input from plasma people about this?)

-- 
David Faure, faure at kde.org, http://www.davidfaure.fr
Working on KDE Frameworks 5