[RFC] [kservice] KPluginMetadata indexing

[Mark Gaiser]

On Thu, Nov 6, 2014 at 4:43 PM, Sebastian Kügler <sebas at kde.org> wrote:
> On Thursday, November 06, 2014 10:09:51 Mark Gaiser wrote:
>> I'm curious about one thing. Have you done some profiling on the
>> current KPluginMetaData to see where the actual hot spot is?
>> In case you don't know how to do that, here are some tips:
>> 1. Recompile Qt with debug symbols (not debug mode, just with the debug
>> symbols) 2. Run a benchmark application via valgrind like so: valgrind
>> --tool=callgrind <your_benchmark_app>
>> 3. Open the output file of the line above in KCacheGrind and hunt for
>> those pesky hot spots.
>>
>> Perhaps there is nothing to optimize and then having an index (and the
>> cost of maintaining it) is worth it, but it would be best to first
>> determine if the current code path can be optimized.
>
> I've focused on reducing the I/O, since that's where we spend by far most of
> the time, somewhere beyond 90% of the whole time it takes to run the query
> (and in worse cases with cold caches and on rotational media, even more).
>
> The next bottleneck would be the deserialization of json data, which is for
> the binary format we use for storage (and I think which is also used in
> QPluginLoader to read the plugins json metadata). As fas as I can see, that is
> also pretty much entirely I/O bound.
>
> There may be something to be gained in the conversion from the KPluginMetaData
> list to the KPluginInfo::List by making the query runner 'understand'
> KPluginMetaData (it knows KService and KPluginInfo currently) but that's
> definitely not a bottleneck currently.
>
> In the whole picture, KPluginMetaData is not a concern right now, reducing the
> I/O is what we need to do first.
>
> Cheers,

Ah right, i think i missed the I/O reducing goal in your initial post.

I haven't checked your draft implementation in detail, but what you
might want to do is:
- Monitor all plugin folders for changes
- If a change in any of the folders is detected: rebuild cache.

That should keep the cache updated in a fairly easy way and remove the
need to have a tool at all.
You could do it more fine grained and only update the cache for the
plugin that changed. It will be a bit more tricky to implement.

Just an idea, but i hope it helps.

Cheers,
Mark