[Kde-pim] Speed of storing large numbers of records, and sizes of akonadiserver and dbus-daemon

[Shaheed Haque]

Hi Andras,

Fetching the pull payload might or might not be an issue (though it
would certainly add overhead!), but I'm not sure what control I have
over the set of items being fetched.

IIUC, the records are fetched in order to determine which items are
new, and which will need updates. It seems to me that the current
logic is designed to handle the case where incremental updates are not
in use, so the whole collection is fetched in order to know what items
need updates/are new. In the incremental case, I would say that only
the items passed in to setIncrementalSyncItems() are relevant to the
check. Anyway, see retrieveGALBatch in:

http://websvn.kde.org/trunk/playground/pim/akonadi/exchange/contacts/exgalresource.cpp?revision=1282168&view=markup

The actual ItemSync job is created around line 844.

Thanks for looking, Shaheed

On 27 February 2012 07:04, Andras Mantia <amantia at kde.org> wrote:
> Hi,
>
>
> Shaheed Haque wrote:
>
>> I managed to catch the inital part of the debug output, and I *think*
>> this is the command that is fetching all the results:
>>
>> akonadi_exgal_resource_269 (0x7fdb30030ec0) 35 FETCH 1:* FULLPAYLOAD
>> ALLATTR CACHEONLY EXTERNALPAYLOAD (UID REMOTEID REMOTEREVISION
>> COLLECTIONID FLAGS SIZE DATETIME)
>>
>> I assume the asterisk is the reason whay all the records are fetched?
>> Assuming so, that seems to come from
>> ItemFetchJobPrivate::startFetchJob(), which is started from
>> ItemSync::doStart(). IIUC, the idea seems to be to fetch the entire
>> collection, modify it, and then write the results using
>> ItemModifyJob. The key line of code seems to be in
>> ItemSync::doStart():
>>
>>   ItemFetchJob* job = new ItemFetchJob( d->mSyncCollection, this );
>>
>> Now, given that I am signalling my intention to modify only part of
>> the collection by calling ItemSync::setIncrementalSyncItems(), how
>> about if the above line was changed to moral equivalent of:
>>
>>   ItemFetchJob* job;
>>   if ( d->mIncremental ) {
>>        incremenatlItems = d->mRemoteItems + d->mRemovedRemoteItems;
>>        job = new ItemFetchJob( incrementalItems, this );
>>   } else {
>>         job = new ItemFetchJob( d->mSyncCollection, this );
>>   }
>>
>> AFAICS, that would be consistent with the definition of
>> setIncrementalSyncItems(), and avoid fetching the whole collection.
>> Any thoughts?
>
> Where is your code, can you give us a weblink or such? I assume somewhere in
> playground, but just point us.
> Maybe you didn't set the setItemSynchronizationFetchScope to not fetch the
> full payload when something has changed?
>
> Andras
> _______________________________________________
> KDE PIM mailing list kde-pim at kde.org
> https://mail.kde.org/mailman/listinfo/kde-pim
> KDE PIM home page at http://pim.kde.org/
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/