[Kde-pim] First step in making akonadi less ressource hungry

[Volker Krause]

On Saturday 14 August 2010 15:55:54 Michael Jansen wrote:
> Currently akonadi uses some kind of broadcast to inform its
> agents/resources/feeders about events. Everyone subscribes to
> one signal NotificationManager::notifiy(...) and then receives every
> single notification. It then decides itself if the item(s) involved is
> interesting. If not it ignores the item.
>
> Since there are quite some subscribers to that signal this means every
> notifications wakes up more than 10 processes that probably only wake up
> to ignore the notification.

correct, I have been thinking about fixing this as well, although my solution 
wasn't as nice and easy as yours as I wasn't aware of the fact that the D-Bus 
daemon is clever enough to not wake up processes that are not connected to a 
broadcasted signal :)

> What initially brought me to that problem is seeing the cpu load when
> importing (synchronizing) a big maildir ressource the first time. At
> that time plasma was really big in the midst of the running. But
> currently it does not register with the messagesource so there is
> probably something i either have to still port or there is another problem.

From what I have seen,  this is only partly caused by the problem described 
above. In the case of the Plasma calendar, the Monitor was configured to 
monitor all collections (recursively starting from root) and only the 
calendar mimetype. With the current Monitor implementation, that will cause 
all messages to match (due to the first filter), ignoring the mimetype filter 
entirely. Additionally, it requests the full payload of monitored objects, 
which results in it downloading *everything*. Same probably happens for the 
invitation feeder and a few others. So, when you add a big maildir, KMail 
only loads headers for the listing when needed, but your panel clocks loads 
the entire folder with all its content.

The evaluation order of the various filters in Monitor was never clearly 
defined, I have a semi-working patch to fix that, makeing most of these 
cases "just work", but it needs a bit more work to make sure there are no 
regressions, the Monitor code is rather fragile.

> How to solve that problem. The first step is to separate the listeners
> from the notification providers point of view. That's exactly what this
> patch is about.
>
> The notificationmanager now only provides two methods:
>
> 1. subscribe( identifier )
> 2. unsubscribe( identifier )
>
> With those two methods anyone interested in Notifications can
> subscibe/unsubscibe with the server. The first method returns a
> QDBusObjectPath. The server registered a new Object there before
> returning the path. Its of type MessageSource and provides currently
> only two methods:
>
> 1. unsubscribe()
> 2. SIGNAL: notify()
>
> The first is a convenience method, the second is the exact same signal
> previously part of NotificationManager.
>
> The patch does not yet solve the problem i explained above. It is just
> the foundation for the solution. Now we have the possibility to let the
> subsciber inform the server about the kind of items they are interested
> in. The server would then first check that information and only emit the
> signal for that particular subscriber if the subscriber told him it
> wants to know about that item type (or told nothing at all).
>
> That's where i need input. What kind of preselection makes sense and how
> should we implements them?
>
> I see
>
> 1. Select by mimetype
> 2. The invitation agent is only interestend in multipart messages. A
> preselection here would make sense two because iiuc the agent has to ask
> for the message before he is able to determine if it is a multipart
> message. 

the invitation agent should probably not listen to general notifications but 
watching a search folder matching its criteria. Otherwise we would need 
type-specific knowledge in the notification filter, which isn't even 
available on the server side.

> 3. ???? 

probably very similar to what Monitor offers right now:
- monitor a set of items
- monitor a set of collections
- monitor a set of mimetypes
- monitor a set of resources
- exclude changes made by ourselves (ie. filter out by session ids)

> For me the patch works but i'm not 100% entirely sure i catched every
> possible subscriber. Are there apps subscribing that do not use Monitor ?

There is at least one, akonadiconsole's notification debugger. However, that's 
not really a problem, we have to keep the old signal anyway for protocol 
backward compatibility (the rules there are very similar to the usual BC 
policy).

Anyway, thanks very much for the work you have done there, I very much like 
the direction this is going into :)
I'd suggest to commit this and cotinue the work in trunk directly, the changes 
look safe enough to me. Just one minor nitpick, maybe MessageSource could 
contain "Notification" as part of its name, "Message" is ambigious in the 
Akonadi/PIM context and could also refer to emails.

Thinking about it, there could be even more optimizations possible once we 
have this in:
- sharing message sources for applications that use the same monitor filters
- automatically filtering by notification type/operation, based on which 
signals are actually connected on the client side

regards
Volker

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-pim/attachments/20100816/1d627362/attachment.sig>
-------------- next part --------------
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/