[Kde-pim] Review Request: fix of resource usage in akonadi-nepomuk-feeder

[Christian Mollekopf]

> On Nov. 30, 2011, 5:07 p.m., Sebastian Trueg wrote:
> > Just so I understand correctly: indexing of this one email takes forever due to the 100 something recipients and thus, you simply ignore the result if it takes longer than 30s? So you start the next storeResources before the other is finished?
> > (Seems to me identification needs improvement)

Indexing the email takes forever (~10s) and thus the KJob (Nepomuk::storeResources) times out with a dbus timeout ("Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken."). Therefore I don't know when the job will actually finish, so I just wait another 30s and then continue the indexing. If the job doesn't timeout I just wait for the job to finish and then send the next batch of data.


- Christian


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://git.reviewboard.kde.org/r/103294/#review8618
-----------------------------------------------------------


On Nov. 30, 2011, 2:38 a.m., Christian Mollekopf wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://git.reviewboard.kde.org/r/103294/
> -----------------------------------------------------------
> 
> (Updated Nov. 30, 2011, 2:38 a.m.)
> 
> 
> Review request for KDEPIM, Nepomuk and Volker Krause.
> 
> 
> Description
> -------
> 
> I sat down and fixed the humongous resource usage of the feeder. The feeder now fetches a hardcoded limit of maximum 100 items with full payload, all other items are queued with their id only. This way this problem should not occur anymore.
> 
> I also managed to reproduce the dbus timeout issue. It looks like even the indexing of a single email can exceed this timeout. Looking at the logs and the indexed data everything looks alright.
> 
> At least for one email I can reproduce this issue, with the small new helper application to index single akonadi items found in nepomukfeeder/test. 
> 
> I suppose the email takes so long because it has ~100 recipients which all have to be identified. However, the timeout is not ideal because it means I cannot wait until nepomuk has finished processing the last batch before sending the next one. My workaround for that is to set a 30s timeout if the datastore job timed out, hope that is enough. This is supposed to avoid that the data just stacks up on the nepomuk side, If you have suggestions for a better solution let me know.
> 
> If this looks okay I will cleanup the whitespace errors/debugging code and commit tomorrow (today actually), before the freeze.
> 
> Sorry for being so late with this stuff.
> 
> 
> Diffs
> -----
> 
>   agents/nepomukfeeder/CMakeLists.txt 903ba667cb51897d21650741fbe34e033ec3792a 
>   agents/nepomukfeeder/feederqueue.h 64676207a3808e8e52557ccaa6abb4cf59acaf68 
>   agents/nepomukfeeder/feederqueue.cpp ecd6c3d30d739b9a3edf0da0f8150312e493ecc8 
>   agents/nepomukfeeder/test/akonadinepomukfeeder_indexer.cpp PRE-CREATION 
> 
> Diff: http://git.reviewboard.kde.org/r/103294/diff/diff
> 
> 
> Testing
> -------
> 
> Works for me.
> 
> 
> Thanks,
> 
> Christian Mollekopf
> 
>

_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/