Folder deletion problem (Re: Problem with imap resource)

[Daniel Vrátil]

On Friday, January 13, 2017 9:47:53 AM CET David Faure wrote:
> On vendredi 13 janvier 2017 09:18:41 CET David Faure wrote:
> > Any idea why only followupreminder_agent is getting a proper ancestor
> > chain?
> Answer: because it's the only one that had the parent in the collectionCache
> (for some reason).
> 
> MonitorPrivate::emitNotification and
> MonitorPrivate::emitCollectionNotification are the problem here.
> 
> When it works, it's because
>         const Collection parent =
> collectionCache->retrieve(colNtf.parentCollection()); found the parent in
> the cache.
> 
> Compare
> akonadi_newmailnotifier_agent(9273)/default
> Akonadi::MonitorPrivate::emitNotification: collectionCache for 948 said 948
> remoteid= "inbox" name= "inbox" with
> akonadi_maildir_resource_0(9267)/default
> Akonadi::MonitorPrivate::emitNotification: collectionCache for 948 said -50
> remoteid= "" name= ""
> 
> After that, emitCollectionNotification simply does:
>     parent = Collection(msg.parentCollection());
> which sets it to id=948 remoteId="", and further down
>    collection.setParentCollection(parent);
> which stores that into the collection that is then passed to
> collectionRemoved().
> 
> This all seems quite fragile, it all depends on what's in collectionCache.
> 
> And MonitorPrivate::ensureDataAvailable says it can't fill that cache in
> case of collection removals.
> 
>     if (msg.type() == Protocol::Command::CollectionChangeNotification
>             && static_cast<const Protocol::CollectionChangeNotification
> &>(msg).operation() == Protocol::CollectionChangeNotification::Remove) {
> //For collection removals the collection is gone anyways, so we can't fetch
> it. Rid will be set later on instead. return true;
>     }
> 
> How is this supposed to work then?

Indeed, this is broken. This is actually broken more than you think, with no 
easy way to fix it completely. It's possible that it worked previously simply 
because maybe an item change event in the collection just before removal 
(maybe triggered when you opened the folder and KMail selected an email in 
there) caused the CollectionCache to be populated. Then you removed the folder 
and it worked. Something might have changed, or the cache gets expired in the 
meantime...and now it's broken :)

Attached is a patch that should fix it a little, please give it a try. In 
Collection removal case we indeed can no longer retrieve the removed 
collection from Akonadi, because it no logner exists in the DB, so the patch 
below at leats tries to retrieve the parent collection and it's ancestor 
chain. Thanks to this the collectionCache->retrieve(colNtf.parentCollection()) 
call in emitCollectionNotification() should succeed and return a full ancestor 
chain. I did not test the patch myself though.


The bigger problem is when you delete a Collection with sub-collections. The 
server will emit the notifications starting with the outter-most collections 
going back to the parent you actually removed, but by the time the 
notification about the leaf collection gets to a Monitor in a client and the 
Monitor requests its parent collection, the parent collection might not longer 
exist either in the server either and then it fails again.

The only real fix for all these problems is Notification Payloads, i.e. 
sending the actual changed entities as part of the notification instead of 
Monitor retrieving them. That's the only way to correctly "snapshot" the state 
of things at the time the notification was created (and it does improve 
performance quite a bit). Maybe I should move it up on my todo list 
again...but for now this patch should fix the most immediate problem.

Dan


-- 
Daniel Vrátil
www.dvratil.cz | dvratil at kde.org
IRC: dvratil on Freenode (#kde, #kontact, #akonadi, #fedora-kde)

GPG Key: 0x4D69557AECB13683
Fingerprint: 0ABD FA55 A4E6 BEA9 9A83 EA97 4D69 557A ECB1 3683
-------------- next part --------------
A non-text attachment was scrubbed...
Name: akonadi-monitor-col-removal-fetch-ancestor-chain.patch
Type: text/x-patch
Size: 1049 bytes
Desc: not available
URL: <http://mail.kde.org/pipermail/kde-pim/attachments/20170113/db524ab0/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-pim/attachments/20170113/db524ab0/attachment.sig>