[Kde-pim] fixing mbox - general Akonadi problem ?

[Ingo Klöcker]

On Tuesday 11 February 2014 19:15:44 Martin Koller wrote:
> On Tuesday 11 February 2014 18:34:22 Kevin Krammer wrote:
> > On Tuesday, 2014-02-11, 18:16:53, Martin Koller wrote:
> > > Hi all,
> > > 
> > > I'm digging into the problems I have with the mbox resource and I
> > > hit
> > > something which might be a general Akonadi problem - or I just
> > > don't know how to:
> > > 
> > > The mbox resource sets a remoteId for each mail which is of the
> > > format CollectionId]::[RemoteCollectionId]::[Offset]
> > > 
> > > Offset is the starting byte position inside the mbox file.
> > > 
> > > When the resource needs to remove a mail permamently from the
> > > file, it needs to rewrite the purged file to disc - which leads
> > > to the problem that ALL remoteIds below the removed mail are
> > > getting invalid!
> > > 
> > > AFAIK Akonadi works asynchronously so added mails to the mbox
> > > (from an external prog) and filtering the mail (moving them to
> > > some other folder) leads to the problem that
> > > MboxResource::itemRemoved() is called with a remoteId which
> > > points to some illegal position in the mbox file. (mbox throws
> > > some error, but then some mails stay in the mbox file whereby
> > > they should be gone leading to duplicate mails on the next load
> > > of the file)
> > > 
> > > How can this be solved ?
> > > How can I globally tell Akonadi when one remoteId changes, some
> > > others need to change, too - in a way so that I can be sure that
> > > no other Akonadi client is already or still using the now
> > > obsolete remoteIds ?
> >
> > You need to modify the items. Basically fetch basic items from
> > Akonadi and modify those which change.
> 
> I tried that - didn't work (but it's easy possible that I made
> something wrong). So i thought that the problem is the async nature
> of
> akonadi. I thought the flow is as follows. Please correct me if I'm
> wrong:
> 
> - external prog fills mbox
> - mbox resource reads new mails and sends new remoteIds to akonadi
> - filter agent gets these and filters (tells mbox resource to remove
> the remoteIds) but when the first to-delete remoteId is received in
> mbox, I already need to modify the others, but the filter agent
> already got also the other ones, so it will still send me the
> old-invalid remoteIds

I don't understand why any remoteIds need to be modified when some 
messages are marked as deleted (by the filter agent)? The remoteIds 
shouldn't change until the mbox is compacted.

The only critical operation on the mbox should be compaction. While a 
compaction is ongoing any write access to the mbox has to be prevented. 
If an external write access during compaction is detected, then the 
compaction needs to be roled back. All remoteId updates to the 
collection should to be done in a single database transaction to prevent 
invalid remoteIds. There is still a tiny chance for inconsistency 
because replacing the old mbox file with the new compacted mbox file and 
updating the remoteIds cannot be done in a single atomic operation, but 
other than that mbox compaction should be safe.


Regards,
Ingo
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-pim/attachments/20140211/a6f0a5ca/attachment.sig>
-------------- next part --------------
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/