[Kde-pim] fixing mbox - general Akonadi problem ?

[Daniel Vrátil]

On Wednesday 12 of February 2014 18:22:40 Martin Koller wrote:
> On Wednesday 12 February 2014 17:41:26 Daniel Vrátil wrote:
> > When you delete a mail from MboxResource::itemRemoved() and there are more
> > items waiting for deletion in ChangeRecorder's pipeline, it's necessary to
> > modify RIDs of all emails affected by the compaction BEFORE
> > changeProcessed() is called. This should cause all other items pending
> > deletion in
> > ChangeRecorder's pipeline to be refetched from Akonadi with updated RID.
> 
> I tried that but that still fails because I try to modify all other moved
> entries via the following code where I want to retrieve the other Akonadi
> Items via an ItemFetchJob but obviously during the exec() event loop, I
> just get the next call to MboxResource::itemRemoved() :-(
> 
> Is that completely wrong what I do here ?
> 
>     // change all remoteIds of the moved items (all items after the deleted
> one change their offset in the file)
>   if ( !movedEntries.isEmpty() ) {
>       qDebug() << "#movedEntries=" << movedEntries.count();
>       Item::List movedItems;
>       const QString colId = QString::number( mboxCollection.id() );
>       const QString colRid = mboxCollection.remoteId();
>       foreach (const KMBox::MBoxEntry::Pair &p, movedEntries) {
>         Item movedItem;

If it's not too difficult (and slow) to extract Message-ID headers from each 
message, you might want to use setGID() instead of setRemoteId(). The reason 
is that in the database the GID column has an index, while remoteId column 
does not, so looking up the items based on GID will be faster.

>         movedItem.setRemoteId( colId + QLatin1String("::") + colRid +
> QLatin1String("::") + QString::number( p.first.messageOffset() ) );
>	      movedItems << movedItem;
>       }
>       ItemFetchJob *fetchItemsJob = new ItemFetchJob( movedItems );

You can optimize the fetch scope a little:
		fetchItemJob->fetchScope().setCacheOnly(true);
		// Don't fetch MTIME attribute
		fetchItemJob->fetchScope().setFetchModificationTime(false);
		// Don't fetch remote ID and remote revision attributes (you don't 
		// need them now)
		fetchItemJob->fetchScope().setFetchRemoteIdentification(false);

>       fetchItemsJob->setCollection( mboxCollection );
>       if ( !fetchItemsJob->exec() ) {
>         cancelTask( i18n( "Could not fetch ids of items to be moved: %1",
> fetchItemsJob->errorString() ) ); return;
>       }
> 
>       qDebug() << "#movedEntries=" << movedEntries.count() << " fetched
> items=" <<fetchItemsJob->items().count();

*theoretically* (and I'm really not sure about it) - some items could have 
been deleted on server in the meanwhile, causing the fetch job to return fewer 
items

>	 Q_ASSERT( movedEntries.count() == fetchItemsJob->items().count() ); 
>	 for (int i = 0; i < movedEntries.count(); i++) {
>         Item itemToMove = fetchItemsJob->items()[i];
>         qDebug() << "item to move:" << itemToMove.remoteId() <<
> itemToMove.id(); const KMBox::MBoxEntry::Pair &p = movedEntries[i];
>         itemToMove.setRemoteId( colId + QLatin1String("::") + colRid +
> QLatin1String("::") + QString::number( p.second.messageOffset() ) );
>		qDebug() << "item to move new rid:" << itemToMove.remoteId();
>		movedItems[i] = itemToMove;
>       }
> 
>       changesCommitted(movedItems);   // is that correct ?

Yes, it should be OK. ResourceBase::changesComitted() internally starts 
ItemModifyJob with all the items and when it finishes, it calls 
changeProcessed() to start a next task. I think however that it would be more 
clear to call the ItemModifyJob manually from here and then just call 
changeProcessed() when it finishes.

Dan

> 
> > Porting MBox resource to ObserverV3 which supports batch deletions and
> > flags changes could help too. It should speed up the resource notably,
> > because it could do compaction after the entire batch is processed,
> > preventing ChangeRecorder from flushing and refetching the entire cache
> > for pipelined notifications after every single item being processed.
> 
> will have a look into this when deletion finally works.

-- 
Daniel Vrátil | dvratil at redhat.com | dvratil on #kde-devel, #kontact, #akonadi
KDE Desktop Team
Associate Software Engineer, Red Hat, Inc.

GPG Key: 0xC59D614F6F4AE348
Fingerprint: 4EC1 86E3 C54E 0B39 5FDD B5FB C59D 614F 6F4A E348
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-pim/attachments/20140212/d407cea6/attachment.sig>
-------------- next part --------------
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/