Akonadi items with duplicate remote ID

Sat Nov 25 14:00:09 GMT 2017

On Wednesday, 22 November 2017 09:32:33 CET David Faure wrote:
> [re-adding kde-pim since it was cc'ed initially]
> 
> On mardi 21 novembre 2017 14:53:39 CET Daniel Vrátil wrote:
> > On Monday, 20 November 2017 10:28:40 CET you wrote:
> > > My kolab resource fails to sync a calendar due to a duplicate RID:
> > > 
> > > Akonadi::Server::AkAppend::parseStream:   ID: 531863 , RID: "1133" ,
> > > GID:
> > > "" , Collection: "Calendar" ( 37 )
> > > Akonadi::Server::AkAppend::parseStream: ID: 586695 , RID: "1133" , GID:
> > > "0088faac-2872-4e34-862b-889bd269f678" , Collection: "Calendar" ( 37 )
> > 

One thing I just realized is that there's probably some other issue behind the 
scenes: why is the Kolab resource suddenly trying to do a RID merge on a 
calendar folder, where it should ALWAYS perform a GID merge? The situation 
that you have a duplicate RID with two different GIDs occured because the 
second Item (with the valid GID) was created by GID merge (no GID match -> 
created a RID duplicate).  If the Resource was doing a GID merge again, the 
conflict wouldn't have occurred since the GID is unique here. The only reason 
why you are seeing the MMC error is because for some reason the resource 
decided to do a RID merge instead of GID merge. I suspect the mix of RID and 
GID merging might've lead to the duplicates in the first place...

> > This is a particularily weird case: you have the same item with the same
> > RID twice, but once with GID and once without. This means that we likely
> > once failed to get the email content from the server (maybe Kolab error?)
> > and extract GID, so the merging failed.
> 
> I've seen this error many times, on different computers, about the same
> shared folder, so I'm tempted to think that it's a reproducible error, but
> we'll see. I had a patch from you with additional debug output but
> unfortunately it was stashed away (and anyway this is about the initial
> sync that brought this problem I guess, too late now).
> 
> > I suppose the first step to mitigate
> > this issue would be for the merging code to disallow empty GID merge and
> > to
> > actually merge if an item with the same RID exists but has empty GID.
> 
> That sounds good, let's do that.
> This requires additional if()s in akappend.cpp and a query that deletes an
> item, right? I can look into it but I'm sure you can write that 10 times
> faster ;)
> 
> Or did you mean doing this in the janitor?
> 
> > Duplicate GIDs are a real thing sadly - for example Kalle's calendar
> > contains a duplicate of an event with the same GID, so it's virtually
> > impossible to sync his calendar :-(
> > 
> > I don't have a viable workaround for this yet - we use GID and not RID for
> > merging explicitly on Kolab calendar folders, since every change to an
> > event means that the old IMAP message with the Kolab object is deleted
> > and the updated Kolab object is appended as a new IMAP message, so the
> > message UID changes, while GID should remain stable. Of course breaks
> > once you have multiple events with the same GID in a folder. I don't know
> > how to easilly solve this yet. We'd need an API in ResourceBase for
> > ItemSync to report back Merge Error so that Kolab could fall-back to
> > merging by RID for that folder, at the cost of slower sync.
> 
> Can't ItemSync take care of that all by itself?
> 
> > > It seems very weird to me to detect duplication and then only clean it
> > > up
> > > in one special case where a mimetype is wrong. Surely fsck should also
> > > clean up the case of duplication with correct mimetypes? No idea how it
> > > should pick the item to keep though.
> > 
> > That's why only the mimetype special case is handled - in that case we can
> > reliably figure out which of the two does not belong. For your case, I
> > don't know what heuristics we could employ to pick the right item to keep
> > or delete.
> 
> Well, we should delete the one with the empty GID, as you said.
> 
> > > Well, in my case, one has a GID, the other doesn't, I suppose it should
> > > delete the one without a GID? See attached screenshot to see all fields
> > > from pimitemtable.
> > 
> > Considerng they both have the same RID and the same size, it's safe to say
> > the are both the same event, so deleting the one without GID is the right
> > step.
> 
> The thing I was wondering was, do we need to "cascade" that deletion, i.e.
> to also remove rows from other tables that would point to that item.
> I see that the janitor code doesn't do that. Doesn't that leave dangling
> attachments for instance? Is there a better method to call for properly
> deleting pim items? It would be rather bad for the janitor to leave a mess
> :-)

-- 
Daniel Vrátil
www.dvratil.cz | dvratil at kde.org
IRC: dvratil on Freenode (#kde, #kontact, #akonadi, #fedora-kde)

GPG Key: 0x4D69557AECB13683
Fingerprint: 0ABD FA55 A4E6 BEA9 9A83 EA97 4D69 557A ECB1 3683
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-pim/attachments/20171125/791c4124/attachment.sig>