[Nepomuk] Redundant DBus communication

Tue Jan 22 11:02:16 UTC 2013

On Sun, Jan 13, 2013 at 6:26 PM, Vishesh Handa <me at vhanda.in> wrote:

>
>
>
> On Sun, Jan 13, 2013 at 5:57 PM, David Faure <faure at kde.org> wrote:
>
>> On Sunday 13 January 2013 17:12:15 Vishesh Handa wrote:
>> > You're right. We can possibly remove the added and removed signals and
>> > infer that data from the changed signal.
>>
>> Won't this require expensive lookups on the receiver side?
>> (If not, then I agree.)
>>
>> The alternative would be to emit added without emitting changed right
>> away --
>> but I don't know the current design (= what happens with these signals).
>>
>
> I'll put up a patch tomorrow.
>

Fixed. Still testing it a bit more before pushing.

>
>
>>
>> > They'll each have their own. This has mostly been done cause it is
>> really
>> > hard to implement it otherwise.
>> >
>> > On the server side we have 3 hash tables which maps the following keys
>>  -
>> > * Resources
>> > * Properties
>> > * Types
>> >
>> > to each ResourceWatcher.
>>
>> Yes, and I wonder if it happens often that the exact same set of keys is
>> requested by the same code in multiple applications.
>> In kdepim, I suspect this might happen? Or maybe I don't really
>> understand the
>> context. It's worth checking, at least.
>>
> > Each ResourceWatcher can watch a number of different combinations of all
>> of
>> > these (They conditions are ANDed). Therefore it is just a lot simpler
>> for
>> > each for each ResourceWatcher to listen to their own dbus path, instead
>> of
>> > creating special dbus paths for the intersection of resource/properties
>> and
>> > types.
>>
>> Right. I'm not suggesting intersections, but equality tests.
>>
>
>
> I've grepped the entire kdepimlibs, kdepim and kdepim-runtime for
> ResourceWatcher. I fond the following -
>
> vlap:~/kde/src/kdepim $ a ResourceWatcher
> messagelist/storagemodel.cpp
> 39:#include <Nepomuk2/ResourceWatcher>
> 122:  Nepomuk2::ResourceWatcher *watcher = new
> Nepomuk2::ResourceWatcher(this);
>
> mailcommon/filter/filtermanager.cpp
> 30:#include <Nepomuk2/ResourceWatcher>
> 122:  Nepomuk2::ResourceWatcher *watcher = new
> Nepomuk2::ResourceWatcher(this);
>
> kmail/tagactionmanager.cpp
> 39:#include <nepomuk2/resourcewatcher.h>
> 62:  Nepomuk2::ResourceWatcher* watcher = new
> Nepomuk2::ResourceWatcher(this);
>
> The FilterManager and TagActionManager are actually both just watching for
> the type nao:Tag. They could indeed be merged together. This would have be
> done on the server side. I'm not sure exactly how to do it, but it is
> possible. I'll try.
>
>
>>
>> If the same very very old email of mine is watched by more than one
>> process,
>> I'm pretty sure it's not related to my own actions, but to some generic
>> watching, which would be the same in multiple PIM processes.
>>
>
> I'm not sure who is watching for emails. It would be nice to know. The
> simplest way would be to add debug statements in the storage service
> (resourcewatchermanager.cpp). I just looked at the code, and it can easily
> be optimized quite a bit as well.
>
>
>> > > I think using dbus is fine for notifications, but still, using it
>> > > efficiently might be better :)
>> > >
>> > > I see nepomukservices using 100% CPU most of the time since I last
>> updated
>> > > kdepim stuff,
>> > > and gdb says it's mostly busy preparing dbus messages to send out.
>> >
>> > Hmm. That's not good.
>>
>> That's mostly due to a bug in qdbusintegrator.cpp though (no cleanup of
>> old
>> dbus objects in the tree, cf my mail to Thiago and you).
>>
>
> Lets see if something can be done in time for 4.10
>
>
>> > But I don't  think it would be simple to change the number of signals
>> being
>> > emitted. One thing that I have been thinking about it the use of
>> > ResourceWatcher inside the Resource class. Though it keeps the cache up
>> to
>> > date, is it really worth it? Maybe we shouldn't do that add a
>> > Resource::reload() function which reloads its contents. The client can
>> > control that manually, if they really require updates.
>>
>> The client wouldn't know when to call reload().
>
>
> They could always create their own ResourceWatcher!
>
>
>> However, if there are use
>> cases where updates are unlikely or unimportant, maybe a
>> Resource::setAutoUpdating(false) would be good to have.
>>
>
> This seems like a better approach. But that would be adding a new API and
> I'm not sure if I should be asking for another exception from the Release
> Team. I'm sure they are quite tired of me. Though most of the applications
> don't really need live updates.
>
> Not sure what to do.
>
>
>> But I really wonder who could possibly be watching my very old email; this
>> must be some generic "watch all emails" thing? I guess I should add debug
>> output in ResourceWatcher, but 1) it's used very very often in kmail, and
>> 2)
>> nepomuk resource URIs don't tell me much.
>>
>>
> Right. Then it is definitely coming from the Resource class.
>
> Try commenting out the details of the
> Nepomuk2::ResourceManagerPrivate::addToWatcher function.
>
>
>> > I think I should run the storage service in callgrind and see what is
>> > actually going on. I've only done that for the file indexer so far, and
>> > that was without any watchers.
>>
>> Well, profiling at the low-level is kind of what I just did.
>>
>> What would be more useful IMHO would be a higher-level picture (who is
>> watching what, and for what purpose), to detect "stupid behavior" at the
>> semantic (no pun intended) level.
>>
>
> Yes. That would be nice.
>
> We need to do a rewrite of the entire NepomukShell and make it actually
> return useful information like this. The storage service does know about
> all of this "higer-level" stuff.
>
>
>> David, who just realized his dbus service IDs are now up to :1.114565 --
>> something is registering/deregistering very very often to dbus... this
>> didn't
>> use to be the case, hmm. Maybe threads coming and going, with their
>> per-thread
>> dbus-connection...
>>
>> --
>> David Faure, faure at kde.org, http://www.davidfaure.fr
>> Working on KDE, in particular KDE Frameworks 5
>>
>>
>
>
> --
> Vishesh Handa
>

-- 
Vishesh Handa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/nepomuk/attachments/20130122/7abf16e7/attachment.html>