[Kde-pim] Speed of storing large numbers of records, and sizes of akonadiserver and dbus-daemon

[Milian Wolff]

On Thursday 23 February 2012 18:50:45 Shaheed Haque wrote:
> Hi,
> 
> I've been doing some tests pushing large numbers of records into
> Akonadi, after noticing some rather big variances in the time taken to
> push 500 Contacts. Before diving into numbers, it is worth saying that
> this set of numbers is from one set of experiments - the numbers from
> other runs do vary wildly, but only in a bad way. Also, I've tried
> extended runs with virtuoso disabled, and that *seems* to make little
> difference, at least to a first approximation.
> 
> - When I started, I was spending more time fetching the data from
> Exchange (3s) than writing it to Akonadi (about 1s).
> 
> - After getting through 20% of my 466k records, the Akonadi write time
> was around 30s (Exchange fetching still at 3s).
> 
> - After getting through 33%, the Akonadi write time is about 32s
> (Exchange at 3s).
> 
> - After 38%, the Akonadi time is about 70 s (Exchange at 3s).
> 
> (System restart)
> 
> - After 39%, the Akonadi time is about 70 s (Exchange at 3s).
> 
> - After 51%, 91s.
> 
> - After 66%, 121s.
> 
> As you can see from the attached graph, the time increases linearly
> with the number of records added. I wonder if we are missing an index
> on a table somewhere? Also, if I were to try to artificially dive the
> records into Contact subfolders, is that likely to help? FWIW, this is
> all with Kontact 4.8.0 on Kubuntu oneiric.
> 
> Anyway, because of this issue, so far, I've never actually managed to
> get an entire set of records for my GAL down from Exchange :-(, so
> input welcome.

This looks bad, but can you point to any code that could trigger this issue? 
Try to use some profiler that allows attaching to processes e.g. to find this 
out - like oprofile, perf, systemtap, vtune, ...

bye
-- 
Milian Wolff
mail at milianw.de
http://milianw.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-pim/attachments/20120224/be0182f8/attachment.sig>
-------------- next part --------------
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/