GSoC: DataEngine caching progress.

[Marco Martin]

I already discussed those points with Brian, so this answer is more as an 
informative thing for everybody else ;)

monday if for Brian and Bruno is ok, we'll have our next weekly meeting 1:30 
pm UTC, if somebody else wants to join, is welcome to do so :)

On Saturday 26 June 2010, Brian Pritchett wrote:
> Hi,
> 	So here is a quick update on the progress of the DataEngine caching
> progress. Right now, I have a service called Storage with a simple API that
> can retrieve and save key/data pairs. Only strings and ints can really be
> supported. However, if the datatype isn't a string or an int, it gets
> converted to a base64 string, and passed. This allows to support any
> datatype.

by using Service, i don't see any ways to do it, so if we stick to service i 
think this will be the final face of the api.

> I've edited DateEngine and have added a setEnableCache(QString, bool) to
> allow any data engine to easily cache a named source. All this member does
> is call DataContainer::setEnableCache(true).

We'll need api there becase we really only want this on dataengine that fetch 
data from the network, imagine how bad would be caching on the time dataengine 
:)

> DataEngine can now call to cache any source, at any time. It will only
> proceed if the DataContainer marked to be cached, and is also marked that
> the data in it hasn't been cached (this is done in setData).
> 
> I was able to easily make the microblogging plasmoid cache its
> TimelineSource by just calling setEnableCache(true) in its constructor. :)
> 
> Things to do:
> * Add a timer to cache all the sources that need to be cached, probably
> every 2-5 minutes.

we decided this way for 2 reasons:
caching each setData() would cause too mucj overhead (even if disk writes 
would be delayed anyways)
caching only at exit, there is risk to lose data due sudden network loss or 
plasma crash.
now ther is the problem when to actually write the dat (probably kconfig won't 
be the final backend, but let's get things right from the beginning)
it cn be immediately after the caching with a sync(), since it would be 
delayed by a timer anyways()
or, as the plsma config, could be the sync() delayed with the timer, and 
always cache immediately at setData() (a bit concerned on the overhead due to 
the base64

> * Add the code that retrieves the data and inserts it when a source is
> created.
> * Currently, the cached data is saved on disk using KConfig, this needs to
> change to saving in Akonadi.

those are the things we discussed at our last meeting, if someone has 
something to add/correct please say so :D

Cheers, 
Marco Martin