ItemFetchJob / ItemCreateJob Answer ?

[Daniel Vrátil]

On Thursday, April 27, 2017 8:11:57 PM CEST Martin Koller wrote:
> Hi,

Hi,

> 
> analyzing the problem with the "multiple merge candidates" I see that when
> kmail asks for a mail to be displayed, it does this via an ItemFetchJob,
> however, maildir answers to the retrieveItems() request with an
> ItemCreateJob in ResourceBase::itemsRetrieved(), which I find strange.

It's ItemCreateJob, but it's executed in the merge mode, which means that the 
server will try to find an existing Item with matching RID (or GID, depends on 
the merge mode) in the given Collection and will merge the new changes into 
the existing Item. If there's no such Item, it will create a new one. This 
makes ItemSync more efficient, because the ItemSync in the Resource no longer 
has to retrieve all known Items from Akonadi just to figure out if it should 
use ItemCreateJob or ItemModifyJob for the new Items passed from the Resource.

The "Multiple merge candidates" error (MMC) then is an error that occurs when 
server tries to find an existing Item to merge into and finds more than one.

> Isn't there a possibility to answer just with some "here is the mail"
> response without even getting into the problem in the akonadi server of
> having to check if there are "multiple merge candidates" ?

itemsRetreived() passes the Items to the ItemSync, which then does its magic 
as desribed above. Theoretically, if the Item ID is known, ItemSync could use 
ItemModifyJob instead of ItemCreateJob, and fallback to ItemCreateJob (in 
merge) mode only if ID is not known.

Maybe even better, as a server-side optimization, the AKAPPEND handler could 
perform merging based on ID if ItemID is present in the command and fallback 
to GID/RID merging otherwise. This would be more efficient, because querying 
Items from DB based on ID is faster than RID/GID queries (because of DB 
index). 

> Of course this would not solve the initial problem of having these multiple
> entries in the DB

Indeed, the best solution would be to fix the problem that leads to the RID 
duplicates in the first place...:-)

> , but at least when kmail asks for an id, which is in the
> DB, which the resource can also read from disk, then I find it at odds that
> the akonadi server prevents the answer being sent back to kmail, just
> because this mechanism is trying to create/merge the item, which it should
> not need to.

One of the nice things about Akonadi is that we are re-using lots of code 
paths for similar tasks, like using the same code for creating new Items as 
well as "updating" existing records. I don't think that having another 
mechanism for Resource to upload bodies to Akonadi would solve the problems.

I have some plans on how to further optimize this though - one plan is 
something I call "Freign Payload" - basically for maildir we would store in 
Akonadi the name of the actual maildir file, so that even if the real body 
would not be stored in Akonadi, KMail could still access it by directly 
reading the file from the maildir. This has some caveats though especially when 
it comes to moving such Items around because it's hard to ensure that the file 
in the maildir exists long enough for everyone who needs to read can read it 
before it's moved.

Second approach is what I call "Direct Payload" and it would basically be 
KMail talking directly to the Resource and requesting the payload directly 
from the resource. The Resource would give back the raw data directly to 
KMail, thus completely by-passing Akonadi. This is more reliable, would also 
provide much more efficiency for mbox and mixedmaildir resources (for which the 
first solution does not work), 

The second approach would require much more work and effort to implement than 
the first one, so for now I'm going with plan A, hopefully for 17.08 or 17.12, 
depending on how fast I can implement Notification Payload, which is a pre-
requisit.

Dan

-- 
Daniel Vrátil
www.dvratil.cz | dvratil at kde.org
IRC: dvratil on Freenode (#kde, #kontact, #akonadi, #fedora-kde)

GPG Key: 0x4D69557AECB13683
Fingerprint: 0ABD FA55 A4E6 BEA9 9A83 EA97 4D69 557A ECB1 3683
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-pim/attachments/20170427/4f73a275/attachment.sig>