[Kde-pim] Speed of storing large numbers of records, and sizes of akonadiserver and dbus-daemon

[Shaheed Haque]

I managed to catch the inital part of the debug output, and I *think*
this is the command that is fetching all the results:

akonadi_exgal_resource_269 (0x7fdb30030ec0) 35 FETCH 1:* FULLPAYLOAD
ALLATTR CACHEONLY EXTERNALPAYLOAD (UID REMOTEID REMOTEREVISION
COLLECTIONID FLAGS SIZE DATETIME)

I assume the asterisk is the reason whay all the records are fetched?
Assuming so, that seems to come from
ItemFetchJobPrivate::startFetchJob(), which is started from
ItemSync::doStart(). IIUC, the idea seems to be to fetch the entire
collection, modify it, and then write the results using
ItemModifyJob. The key line of code seems to be in
ItemSync::doStart():

  ItemFetchJob* job = new ItemFetchJob( d->mSyncCollection, this );

Now, given that I am signalling my intention to modify only part of
the collection by calling ItemSync::setIncrementalSyncItems(), how
about if the above line was changed to moral equivalent of:

  ItemFetchJob* job;
  if ( d->mIncremental ) {
       incremenatlItems = d->mRemoteItems + d->mRemovedRemoteItems;
       job = new ItemFetchJob( incrementalItems, this );
  } else {
        job = new ItemFetchJob( d->mSyncCollection, this );
  }

AFAICS, that would be consistent with the definition of
setIncrementalSyncItems(), and avoid fetching the whole collection.
Any thoughts?

Thanks, Shaheed

On 25 February 2012 21:56, Shaheed Haque <srhaque at theiet.org> wrote:
> Understood. But even with the debugger disabled, I'm back at the sorts
> of numbers I reported in an earlier thread:
>
> 49 ms versus 63655 ms versus 3141 ms
>
> So my question stands: are the hundreds of thousands of records that
> are being fetched in order to update 2 records only there because I am
> running the debugger, and if not, then surely that is not right?
>
> On 25 February 2012 21:32, Andras Mantia <amantia at kde.org> wrote:
>> Shaheed Haque wrote:
>>
>>> Sure, that was just a debug session. But isn't the important thing all
>>> the data it is fetching, or is that affected by the debugger too?
>>
>> If you have the debugger enabled and you communicate with akonadi server in
>> any way, that communication is sent to the akonadiconsole, that displays it,
>> etc. The whole behavior is much slower and your numbers are compromised.
>>
>> Andras
>> _______________________________________________
>> KDE PIM mailing list kde-pim at kde.org
>> https://mail.kde.org/mailman/listinfo/kde-pim
>> KDE PIM home page at http://pim.kde.org/
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/