Plasma Plugin Loading

[Sebastian Kügler]

Hi,

I've seen that Aaron asked about the way we want to load plugins, and pointed 
out a number of different ways this is currently done. Since I've worked a bit 
on these bits last year, I thought I'd give some background information and 
propose a strategy forward -- nothing set in stone, but as a base for further 
discussions. It does, at least, make sense to me. :)

TL;DR:
- the transition from KService / ksycoca towards KPluginInfo is not finished 
yet
- for binary plugins, we do not need KService / ksycoca anymore, metadata is 
queried directly from the plugin.so file's metadata section
- the APIs necessary to simplify Plasma::PluginLoader and our installation of 
non-binary plugins are there, we should be able to transition away from 
ksycoca "pretty easily" by querying the metadata.desktop files inside the 
packages directly


First, quick background. Qt changed the way plugins are loaded in Qt 5. Our 
old infrastructure has been updated to use Qt's new plugin mechanism 
internally. This allows to bake metadata directly into the plugin's binary and 
load it. This has mostly been a transparent change for creating plugins, since 
we were able to change the macros to create the new Qt plugins. KPluginLoader 
and KPluginFactory are basically handling this transparently.  For loading 
plugins, the KService::List-based APIs have been deprecated, recommended is 
new API in KPluginTrader::query, operating on KPluginInfo instead of KService, 
in essence, KServiceTypeTrader -> KPluginTrader.

The change for plugins is that at build-time, we're now converting the 
.desktop file to json, and bake that information into the plugin through Qt5's 
new standard mechanism. (In Plasma, that's the _WITH_JSON C++ macros and the 
kservice_desktop_to_json() cmake macro handling that.) We do still install the 
metadata.desktop file into the services directory, in order to keep 
KServiceTypeTrader working (see plasma_install* cmake macros).

For now, KPluginTrader is not a complete replacement for all our needs. Two 
things especially:

- It's slower than ksycoca. In order to query metainformation, it needs to 
read it from every plugin. This scales linearly with the size of the plugin 
dir. We're installing plugins into subdirectories in KF5, in part to reduce 
the cost of plugin loading. In Plasma, we install (binary) plugins into 
subdirectories based on their servicetype. I'm not privy to details, dfaure 
has ideas about this. Also, on non-rotational media, this is not much of an 
issue (relatively small amounts of data read, lots of disk seeks). Rotating 
media suck pretty much exactly what one would expect (in the range of 3-7ms 
per plugin queried, disk caches alleviate cases where the same plugins are 
read multiple times. Ultimately, the idea is to create a cache, but not 
ksycoca -- ask dfaure.

- KPluginTrader only loads .so files. Plasma uses a lot of non-binary plugins. 
In these cases, we need the KService-based API (KServiceTypeTrader and 
friends) right now. In Plasma::PluginLoader, we're using KPluginTrader (so 
KPluginInfo-based API) for binary plugins such as DataEngines and 
PackageStructures, and KServiceTypeTrader for non-binaries -- applets and 
packages in general. The transition from KService to KPluginInfo (and thus 
away from ksycoca) there is not finished yet.

So for binary plugins, we have a way now to avoid KService and ksycoca. In 
Plasma, we use KService for two things: querying and loading its KPluginInfo. 
The querying in KPluginTrader can operate on KPluginInfo lists since 5.0 
(KPluginInfo::List KPluginTrader::query(const QString &subDirectory, const 
QString &servicetype, const QString &constraint)). This function walks over 
the installed plugins in a given subdirectory under Qt's plugin pathes 
(QCoreApplication::libraryPaths()), and loads all the plugins to retrieve the 
metadata; creates a KPluginInfo::List from that metadata, then it filters it 
based on the constraints. 

We can now change Plasma::PluginLoader away from ksycoca by creating a list of 
KPluginInfos from all the metadata.desktop files in our installation paths and 
get back a list filtered on constraints. That's basically replicating 
KPluginTrader::query(), to use metadata.desktop files and populate a 
KPluginInfo::List from that, instead of constructing KPluginInfos from 
QPluginLoader::metaData(). This would, I think, be a logical next step.

One thing we might run into here is that our servicetypes have custom keys 
that are not known to KPluginInfo, we're interested in the type information of 
our custom keys, especially. We do still have the service definitions with 
that info, supposedly we can load this from there.

That means we can use a mechanism similar to the Qt one to query plugins and 
then load them, KService would be out of the picture, so would ksycoca be -- 
we can use the query language parser without KService. 
KPluginTrader::applyConstraints(KPluginInfo::List &list, QString constraints) 
is indeed public API. That's the the really complex piece of code. The rest is 
basically looking up files in a bunch of directories.

The upside is that we can then remove the separately installed .desktop files 
in the services directories, and packages would be entirely standalone, no 
ksycoca runs necessary anymore, installing a package is simply a matter of 
unzipping it into the right package directory, removing it also becomes 
easier, and we have less problems with stale or outdated service files around.
Regarding the packages and plugins itself, this is a fully transparent change. 
It also means that plasmapkg can be cut down a bit, likely more cleanups can 
be done in other places. Also, less deprecation warnings. 

I think my brain is empty now. :)

Cheers,
-- 
sebas

http://www.kde.org | http://vizZzion.org | GPG Key ID: 9119 0EF9