[Kde-pim] Review Request: fix of resource usage in akonadi-nepomuk-feeder

[Anders Lund]

On Onsdag den 30. november 2011, Christian Mollekopf wrote:
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://git.reviewboard.kde.org/r/103294/
> -----------------------------------------------------------
> 
> Review request for KDEPIM, Nepomuk and Volker Krause.
> 
> 
> Description
> -------
> 
> I sat down and fixed the humongous resource usage of the feeder. The feeder
> now fetches a hardcoded limit of maximum 100 items with full payload, all
> other items are queued with their id only. This way this problem should
> not occur anymore.

You must be the #1 KDE hero of 2011! <3

> I also managed to reproduce the dbus timeout issue. It looks like even the
> indexing of a single email can exceed this timeout. Looking at the logs
> and the indexed data everything looks alright.
> 
> At least for one email I can reproduce this issue, with the small new
> helper application to index single akonadi items found in
> nepomukfeeder/test.
> 
> I suppose the email takes so long because it has ~100 recipients which all
> have to be identified. However, the timeout is not ideal because it means
> I cannot wait until nepomuk has finished processing the last batch before
> sending the next one. My workaround for that is to set a 30s timeout if
> the datastore job timed out, hope that is enough. This is supposed to
> avoid that the data just stacks up on the nepomuk side, If you have
> suggestions for a better solution let me know.

A side thought: When I receive a spam mail with (lots of) bogus receivers, do 
akonadi/nepomuk attempt to identify all of those?

I think I understood at one point that filtering is now happening AFTER the 
mail have been placed in inbox, and if the above is correct, there is another 
reason to fix that. 

I probably receive more spam than ham, or about the same amount. The spam 
mails will be likely to have most receivers and sometimes lots of 
attaachments, which probably means that they will be the ones slowing things 
down - for no good reason!


> If this looks okay I will cleanup the whitespace errors/debugging code and
> commit tomorrow (today actually), before the freeze.
> 
> Sorry for being so late with this stuff.
> 
> 
> Diffs
> -----
> 
>   agents/nepomukfeeder/CMakeLists.txt
> 903ba667cb51897d21650741fbe34e033ec3792a
> agents/nepomukfeeder/feederqueue.h
> 64676207a3808e8e52557ccaa6abb4cf59acaf68
> agents/nepomukfeeder/feederqueue.cpp
> ecd6c3d30d739b9a3edf0da0f8150312e493ecc8
> agents/nepomukfeeder/test/akonadinepomukfeeder_indexer.cpp PRE-CREATION
> 
> Diff: http://git.reviewboard.kde.org/r/103294/diff/diff
> 
> 
> Testing
> -------
> 
> Works for me.
> 
> 
> Thanks,
> 
> Christian Mollekopf
> 
> _______________________________________________
> KDE PIM mailing list kde-pim at kde.org
> https://mail.kde.org/mailman/listinfo/kde-pim
> KDE PIM home page at http://pim.kde.org/


-- 
Anders
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/