[Kde-pim] Problem with bulk fetching of items with 4.8

[Kevin Krammer]

Hi Shaheed,

I am not entirely sure what problem you are trying to solve.

My interpretation is that Exchange does not allow you to query for items that 
have been created/modified/deleted since the last sync.
Since that would mean there is no such thing as the concept "remote revision", 
why would you want to store one?

Without the option of just getting the updates you will have two set of items, 
one from Akonadi and one from Exchange.

Any item in E but not in A is new.
Any item in E and in A is potentially modified.
Any item not in E but in A has been deleted

But as I said I think I don't understands the problem.

Cheers,
Kevin

P.S.: is this requirement for full list download a limitation in the context 
of contacts or does Exchange also require email clients to full list a folder 
every time?

On Saturday, 2012-02-04, Shaheed Haque wrote:
> Hi Kevin,
> 
> Yes, I've been looking into ItemSync, and as of earlier today, have
> something based on it which seems to work as well as my pre-4.8 code.
> In fact better, since resuming a download seems consistent now
> (possibly as a result of the overall 4.8 improvements?). Anyway, I'm
> now hopeful that if I wait the ~5 hours needed, I'll be able to fetch
> all 466k contacts.
> 
> So, that brings me to my final design problem...
> 
> Exchange does not seem to have any way for me to track changes, so to
> work out what obsolete records I need to delete, I was thinking of
> using a timestamp (stored using a custom attribute on the Collection),
> and then using a query based on the (creation) "datetime" column of
> the pimitemtable to find the obsolete items. In SQL terms, I think I
> need something like:
> 
>     SELECT itemid FROM pimitemtable WHERE collectionid=<mycollection>
> AND datetime LT <mycutoff>;
> 
> I'd propose to use an ItemSearchJob to run the query...all I need is
> to figure out how to write the sparql version of the above. (I looked
> at the queries in the ContactSearchJob sources, but that didn't get me
> too far :-)). So, any query experts out there?
> 
> Thanks, Shaheed
> 
> 2012/2/4 Kevin Krammer <kevin.krammer at gmx.at>:
> > Hi Shaheed,
> > 
> > sorry for not replying earlier.
> > 
> > On Wednesday, 2012-02-01, Shaheed Haque wrote:
> >> With the hints provided, I now have something reasonably functionally
> >> clean in SVN. Unfortunately, the performance is a bit poor in the
> >> sense that I can fetch 300 items from Exchange in around 1000 ms, but
> >> it then takes me ~60,000 ms to store in Akonadi. Now, given that I'm
> >> writing one item at a time to Akonadi, the obvious thought is that it
> >> would be better to run all the writes within the context of a
> >> transaction, and use a single commit() at the end (as per the model I
> >> started this design with). However, I cannot make that work...
> >> 
> >> Basically, that approach seemed to work in the early stages when I
> >> simply used ItemCreateJob on each item, but (somewhat to my surprise),
> >> I found that creating the same object twice (i.e. with the same
> >> remoteId) resulted in two objects being created. So, I end up trying
> >> to ItemFetchJob first and depending on the success or failure,
> >> ItemModifyJob or ItemCreateJob.
> > 
> > The maildir resource's retrieveitemsjob basically uses the same approach.
> > However, I think it first fetches all current items and the compares to
> > that job's result when processing the remote items.
> > 
> >> At this point, I'm a bit unclear what options I have. I *think* I'd
> >> like to be able to do whatever it is that ItemsRetrievedIncremental()
> >> does...
> > 
> > The ResourceBase's implementation for retrieve items methods use a
> > special job class called ItemSync.
> > It is more or less a transaction of item create, modify and delete jobs
> > based on comparison with the result of prior item fetch.
> > I think it is actually a public class.
> > 
> > Maildir resources uses a different implementation because timestamp
> > information on its backend (maildir files) allow it to only process a
> > subset of backend items.
> > 
> > Cheers,
> > Kevin
> > 
> > --
> > Kevin Krammer, KDE developer, xdg-utils developer
> > KDE user support, developer mentoring
> > 
> > 
> > _______________________________________________
> > KDE PIM mailing list kde-pim at kde.org
> > https://mail.kde.org/mailman/listinfo/kde-pim
> > KDE PIM home page at http://pim.kde.org/
> 
> _______________________________________________
> KDE PIM mailing list kde-pim at kde.org
> https://mail.kde.org/mailman/listinfo/kde-pim
> KDE PIM home page at http://pim.kde.org/


-- 
Kevin Krammer, KDE developer, xdg-utils developer
KDE user support, developer mentoring
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 190 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-pim/attachments/20120204/47ff2d3c/attachment.sig>
-------------- next part --------------
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/