[Kde-pim] Nepomuk Feeder Improvements

[Christian Mollekopf]

Hey,

Ok, we'll implement the cache in the feeder.

I'll try to summarize all the issues we have otherwise with the feeders
soon.

Cheers,
Christian

On Sun, Oct 28, 2012, at 05:58 PM, Vishesh Handa wrote:

On Sun, Oct 28, 2012 at 10:05 PM, Christian Mollekopf
<[1]chrigi_1 at fastmail.fm> wrote:

On Sun, Oct 28, 2012, at 02:31 PM, Vishesh Handa wrote:
> On Sun, Oct 28, 2012 at 2:34 PM, Andras Mantia <[2]amantia at kde.org>
wrote:
>
> > Hi,
> >
> >  I have a question about caching. Do you mean that the feeder
should cache
> > in memory all contacts and especially all headers for every mail?
> >
>
> Nope. When a contact is stored in Nepomuk, it is given a unique
> identifier
> of the form 'nepomuk:/res/some-uuid'. I would like to cache this uri
for
> certain regularly used contacts, and headers.
>
> Each header is a separate object in Nepomuk, and has its own
identifier.
> When pushing common headers such as 'Mailing List', it makes sense
the
> cache the last 10 mailing lists identifiers pushed. That way Nepomuk
has
> to
> avoid the job of querying the database to see if an object exists and
> with
> the corresponding information, and finding its identifier.
>
> I hope this makes it clearer.
>


  Hey,
  I understand the intentions and how it's supposed to work, it's not
  entirely trivial though and I'm wondering if we wouldn't be better
  off
  implementing the same feature inside nepomuk.

I cannot do that cause I have no knowledge about the data being pushed.
StoreResources has to be generic. The only way I can identify the
resources is based on its properties.
You on the other hand know exactly what data you're pushing. I think it
would be trivial to map a Mbox to a QUrl.

  Since the cache would grow endlessly, we'd have to expire it's
  values
  based on access, size and maybe also time, given the same data is
  available in nepomuk,

Use QCache, and cache the last 20-30 elements. Shouldn't consume too
much memory.

  I think we could also implement the feature there.
  Of course, by implementing the feature in the feeder we'd have some
  additional information, such as which resource we actually want to
  try
  caching, but given the level of intelligence the cache needs anyways
  I
  think the cache would achieve about the same performance boost by
  just
  caching everything (or a defined set of resources).

Possibly. Though I'm not sure how to do it.

  Do you think this cache could also be useful to other nepomuk
  clients or
  is a very feeder specific solution?

Well, the file indexer doesn't need the cache, and telepathy already
has one.

  Cheers,
  Christian

> If yes, that is probably not good, it would use a lot of memory.
> >
> > Andras
> > _______________________________________________
> > KDE PIM mailing list [3]kde-pim at kde.org
> > [4]https://mail.kde.org/mailman/listinfo/kde-pim
> > KDE PIM home page at [5]http://pim.kde.org/
> >
>
>
>
> --
> Vishesh Handa
> _______________________________________________
> KDE PIM mailing list [6]kde-pim at kde.org
> [7]https://mail.kde.org/mailman/listinfo/kde-pim
> KDE PIM home page at [8]http://pim.kde.org/

References

1. mailto:chrigi_1 at fastmail.fm
2. mailto:amantia at kde.org
3. mailto:kde-pim at kde.org
4. https://mail.kde.org/mailman/listinfo/kde-pim
5. http://pim.kde.org/
6. mailto:kde-pim at kde.org
7. https://mail.kde.org/mailman/listinfo/kde-pim
8. http://pim.kde.org/
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/