EWS resource server-side notification handling problems

[Daniel Vrátil]

On Thursday, 3 October 2019 22:19:56 CEST Krzysztof Nowicki wrote:
> Hi,

Hi!

> 
> When working with the EWS resource I'm experiencing some messy event flow
> that leads to the folder getting out of sync in a fatal way causing the
> need for a full resync. I'm seeking some advice how to fix this.
> 
> The problem happens in case of a modify+delete action (e.g. a message is
> read and quickly deleted).
> 
> So here is what happens:
> 
>  1. The message attributes are modified (e.g. the message is read, causing
> the SEEN attribute to be set).
>  2. Akonadi calls itemsFlagsChanged() for the item, which triggers an
>      UpdateItem EWS request.
>  3. The message is deleted.
>  4. Akonadi calls itemsRemoved() for the item, which triggers a DeleteItem
> EWS request.
> 
> So far so good, but here is where the server-side notification subscription
> kicks in:
> 
>  5. The EWS server sends a notification about the item having been updated
> (in response to the request from step 2. The EWS server notification
> mechanism does not care if the modification originated from the same
> client. 6. The event triggers a collection sync for the folder, which
> retrieves the list of changes on the server side and on the Akonadi side.
>  7. The synchronization code sees that an item has been modified remotely
> and looks for the matching local item in order to update it (at this point
> I don't know that I have just modified the item). The lookup fails as the
> item is already gone (step 3), at which point the code assumes that we're
> seriously out-of-sync and triggers a full sync.
> 
> I have tried to somewhat mitigate this by logging items that were locally
> modified or deleted and in case a notification is received for that
> operation, it is ignored. This reduced the window of opportunity slightly,
> but with the right timing it still happens.
>
> So I was wondering if the logic I use to handle server-side notifications is
> correct. At one time I was thinking whether instead of triggering
> collection syncs, I should let the notification handler apply the changes
> manually by manipulating Akonadi items directly, but that could lead to
> another amplification, as the resource would in turn get calls from Akonadi
> about item operations that it has just made from the notification handler
> thread.

It is safe to do a change to an Item from within the resource, as long as you 
perform this on the default session. The Akonadi server is clever enough to 
not send a change notification to the resource if the change was triggered by 
the resource, since it assumes the resource knows what it did.

This only solves your race condition in part, though. Even when you would try 
to apply the change reported by EWS from the resource back to Akonadi, you 
would get an error if the item was deleted in the meantime. You would probably 
need to try to fetch the item before trying to update it to see if it did not 
disappear, but even that still leaves a small window for a race if the item is 
deleted from Akonadi between the fetch and the modify jobs...I can't think of 
a really air-tight solution to completely avoid races.

- Dan

> I was also thinking about a way to check if the local copy of a folder is in
> sync with the remote one (to try to avoid a costly full sync), but the EWS
> API does not offer any reliable way to compare local and remote states.
> 
> Any thoughts/ideas?
> 
> Regards
> Chris


-- 
Daniel Vrátil
www.dvratil.cz | dvratil at kde.org
IRC: dvratil on Freenode (#kde, #kontact, #akonadi, #fedora-kde)

GPG Key: 0x4D69557AECB13683
Fingerprint: 0ABD FA55 A4E6 BEA9 9A83 EA97 4D69 557A ECB1 3683
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-pim/attachments/20191005/de62e5e5/attachment.sig>