[Kde-pim] Question about notifications+IMAP

[Volker Krause]

Hi,

cc-ing kde-pim for this, I hope you don't mind.

On Wednesday 29 July 2009 04:05:28 Szymon Tomasz Stefanek wrote:
> I've been working on the hidden items as discussed.
>
> I've added the flag to the db and disabled the notifications
> for hidden items in NotificationCollector::itemNotification()
> (seems a nice central place to do it, but you may suggest
> something else here).

can't think of a better place for that

> It works: the appended items are pushed to the preprocessor
> chain which at the end of the job "unhides" the item
> and emits the itemAdded() notification.
>
> However I've found an interesting side effect. The IMAP
> resource tends to "loop" on the last message received.
> That is.. it finds out a new message on server, it
> downloads it, it stores it on the akonadi server,
> then it finds out that a new message "appeared"
> in the local storage, it pushes it to the IMAP server, then
> it finds out again that a new message is on the IMAP server
> etc...

ouch...

> After some debugging I've possibly found an explaination
> but I'll need your confirmation about this.
>
> Actually the Agents instances ignore the notifications
> about the changes they have caused in the Akonadi storage
> by discarding the messages carrying their own session id.

yep, to prevent exactly the problems you are seeing now

> So when a Resource retrieves a new message from the
> server and invokes an Append handler, the itemAdded()
> notification reaches everyone but the Resource itself.
>
> Now I've disabled that notification in the original
> place and emitted it from the PreprocessorManager code
> which "sits" in a session with a different id.
>
> So the Resource now receives the notification about
> the auto-added item and thinks that it has been
> added by someone else: "hey! I must store it on the IMAP server!".
>
> At this point the resource decides that it has to change
> the remoteId of the item it has just uploaded (I assume that
> this is the right behaviour for any locally created message).

yep

> Then some obscure IMAP+Akonadi machinery triggers
> and it ends up that the locally stored remote identifiers
> don't match the identifiers present on the IMAP server:
> the server has one more... a new message to be downloaded
> which restarts the whole loop.
>
> Does this look reasonable as explaination ?
> Or I'm missing something ?

No, unfortunately this is exactly what happns there.

There are even more cases where the hidden items could cause problems: If a 
resource syncs for a second time while the newly created items from the first 
sync are not yet processed, it will likely try to create them again. This 
would even happen if the notifications would have the correct session id.

> Now the question is how to avoid this behaviour.
>
> An idea would be to save the sessionId of the
> creating resource in the Append handler. Then, when the
> preprocessing chain has finished, temporairly fool
> the NotificationCollector used by the PreprocessorManager
> to emit the notification with the saved sessionId.
>
> However this could cause some incoherent results
> if the original Resource is dead in the meantime
> and someone else has taken its sessionId (is this possible ?).

usually that's not possible for resources as their session id matches the 
resource id.

> It also looks a little bit as a "hack".

yep, my feeling as well

> Do you have a better idea ?

on the contrary, thinking about this I find more potential issues in this 
area: assuming a filter modified or moved the item, we also need to emit the 
corresponding change notification afterwards, so the resource can process 
this change. While this might sound easy, there is some semi-clever change 
notification compression going on which will probably drop the following 
modify/move notification if the corresponding add notification is still in 
the queue (which normally is correct).

So, we not only need to record the session id, but also all other change 
notifications that would normally happen while the item is in the 
preprocssing chain.

For the syncing problems noted above, it might help to not hide the item for 
their owning resource. Hm, no, that still can cause problems if the sync sees 
the changes made by some preprocessors and tries to overwrite them.

Another idea might be to not return from an APPEND command unless the 
preprocessing has been completed, effectively making the preprocessing 
synchronous. Not sure what other side-effects that might have, it will 
definitely fail for retrieving more data from the resource though (the IMAP 
load on demand attachments case).

That's all I can come up with for now :-(

regards
Volker



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-pim/attachments/20090729/f37f7981/attachment.sig>
-------------- next part --------------
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/