[Kde-pim] [Differential] [Request, 386 lines] D1636: Implement client-side threading cache

[dvratil (Daniel Vrátil)]

dvratil created this revision.
dvratil added reviewers: mlaurent, vkrause.
dvratil set the repository for this revision to rMESSAGELIB PIM: Message Library.
Restricted Application added a project: KDE PIM.
Restricted Application added a subscriber: kde-pim.

REVISION SUMMARY
  The cache is a simple QHash<ChildId,ParentId> that we build when folder is
  opened for the first time and persist it in a cache file for each folder.
  Aggregation state (grouping and threading) is also stored in the cache
  file so we can check whether the cache matches the current aggregation
  configuration before we load it. If the aggregation has changed we simply
  discard the cache file and perform full un-cached threading.
  
  There is a second QHash<ItemId, Item*> in ThreadingCache which we fully
  populate in Pass1. Pass1 may already yield some perfect matches thanks
  to the Child-Parent cache, but only if the Parent Item* has already been
  inserted into the second QHash - this does not happen much as we retrieve
  Items in reverse order so children will arrive before their parents. If
  we can't do a perfect match but we have the Parent ID available in cache
  we just move the Item to Pass2 and go on to next one. Otherwise we let
  Pass1 do full evaluation which will insert the Item to cache if perfect
  match is found or queue it for Pass2.
  
  In Pass2 we have the second QHash fully populated so we perfect-match
  all Items using the ChildId->ParentId cache. Items that are known to not
  have a parent (i.e. thread leaders) are scheduled for Pass4, Items that
  are not available in cache are sent for full evaluation through Pass2
  (and Pass3 if needed) and inserted into the cache.
  
  Finally Pass4 performs grouping. There is no caching for that right now
  because the grouping is dynamic and there are no real stable identifiers
  for group headers. We could possibly cache all the fixed groups (i.e. sender,
  receiver or subject) and maybe even .fixed-date groups (e.g. "January 2014",
  "March 2015") and only let Pass4 calculate dynamic groups ("May",
  "Two weeks ago", "Yesterday", ...) but the gain here would be minimal as we
  are usually dealing with very few groups. The real bottleneck of Pass4
  is beginInsertRows()/endInsertRows() as threads are shifted around - something
  to look into next.
  
  On my system this speeds up opening a folder with 50000 emails by ~30%.

REPOSITORY
  rMESSAGELIB PIM: Message Library

REVISION DETAIL
  https://phabricator.kde.org/D1636

AFFECTED FILES
  messagelist/src/core/model.cpp
  messagelist/src/core/model_p.h

EMAIL PREFERENCES
  https://phabricator.kde.org/settings/panel/emailpreferences/

To: dvratil, mlaurent, vkrause
Cc: kde-pim, dvasin, winterz, smartins, vkrause, mlaurent, knauss, dvratil
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/