plugins,caches, startup time

[Aleix Pol]

On Fri, Sep 30, 2016 at 2:35 PM, Sebastian Kügler <sebas at kde.org> wrote:
> Hi,
>
> On Tuesday, September 27, 2016 4:52:22 PM UTC Marco Martin wrote:
>> as there is being some work in plugin caching right now, to see if /what we
>> can save at startup time would be nice if different people on different
>> systems do some tests.
>> on his system, by indexing plugins Sebas saves about 2-300 milliseconds. How
>> did you test that, so that we see wether we can reproduce it on different
>> systems, (may well be that on some kinds of hardware it's worth it, not on
>> some other)
>
> My measuring was done as follows:
> - I instrumented Plasma::PluginLoader and recorded all calls to
> KPluginLoader::findPackage(...)
> - I put these queries into an autotest, and instrumented that with timers that
> measure the time spent in KPluginLoader::findPlugins(...), with and without
> cache.
>
> So my tests are a little more synthetic. My code is in kcoreaddons[sebas/
> pluginindexer2], the index is generated with
>
> $ kpluginindexer -u
>
> Results on my fast system (core i7 + ssd) is that for Plasma startup, the
> indexed code runs about 200ms faster (out of a total ~700). That's not a lot
> of gain, IMO, and I'm unsure if it justifies the complexity added by
> maintaining an index.
>
> I haven't done any measurements on cold caches.
>
>> I did a similar test on kpackage plugin indexes, that's how all plasmoids
>> get loaded, so it's quite a lot of them.
>
> Nice, I suppose KPackages are a little different to plugins, since there's
> more overhead reading, the package metadata is stored as .desktop file, which
> means more conversion costs compared to the binary json that's in the plugin
> binaries. Aleix has recently been looking into converting it to json, but that
> would only save the .desktop to json part, binary json is still much quicker
> to read (the format is pretty close to Qt's internal representation in memory,
> deserialization is almost entirely I/O-bound.

It's not only about loading from json. Keep in mind that even adding a
cache we'll be loading each metadata.desktop a number of times either
way (both using KConfig parsing and KCoreAddons' DesktopFileParser).

My suggestion is to first get the type system right in
plasma-framework then we look into enforcing caches. I'm especially
afraid of this one as it will require quite a lot of work from the
packagers.
Furthermore, it would also be really interesting to look into reducing
plugin look-ups altogether, I'm almost sure that plugins are listed
several times on every boot.

Aleix