D7742: KDevelop projects: optional use of KDirWatch and defer it and parser start to after project loading

[René J.V. Bertin]

rjvbb added a comment.


  This new revision also includes temporary (a priori) code to obtain timing estimates and a hack to disable the full defer (dirwatcher creation is always deferred to after the project was imported). I'm including that additional code so others can get some timing estimates from their work flow too.
  
  Here are some timing results on my 2011 MacBook Pro with 2.7Ghz i7, as requested by Sven on https://phabricator.kde.org/D7745:
  
  A session with projects for the 5.1 branches of KDevelop and KDevPlatform + the Purpose project. The non-deferred dirwatcher timings are probably better than what KDevelop achieves now since the current version of the patch creates them after importing the project and not before.
  
  dir watching, no full parse
  `deferred: import = 3.9—4.7 seconds, creating the watchers = 1.4—1.6 seconds` (vs. //0.2 seconds// on a much slower Linux rig!)
  non deferred (debug output):
  
    kdevplatform.shell: Project "purpose-git" imported in 4.405 seconds
    kdevplatform.filemanager: Project dir monitoring is enabled for project "purpose-git"
    kdevplatform.shell:      "purpose-git" : creating dir watchers and/or starting parse jobs took an additional 5.051 seconds
    kdevplatform.shell: Project "kf5-kdevplatform-devel (MP)" imported in 2.261 seconds
    kdevplatform.filemanager: Project dir monitoring is enabled for project "kf5-kdevplatform-devel (MP)"
    kdevplatform.shell:      "kf5-kdevplatform-devel (MP)" : creating dir watchers and/or starting parse jobs took an additional 2.74 seconds
    kdevplatform.filemanager: Project dir monitoring is enabled for project "kf5-kdevelop-5 (MP)"
    kdevplatform.shell: Project "kf5-kdevelop-5 (MP)" imported in 3.792 seconds
    kdevplatform.filemanager: Project dir monitoring is enabled for project "kf5-kdevelop-5 (MP)"
    kdevplatform.shell:      "kf5-kdevelop-5 (MP)" : creating dir watchers and/or starting parse jobs took an additional 3.792 seconds
  
  2nd run:
  
    kdevplatform.shell: Project "purpose-git" imported in 3.193 seconds
    kdevplatform.filemanager: Project dir monitoring is enabled for project "purpose-git"
    kdevplatform.shell:      "purpose-git" : creating dir watchers and/or starting parse jobs took an additional 3.559 seconds
    kdevplatform.shell: Project "kf5-kdevplatform-devel (MP)" imported in 1.481 seconds
    kdevplatform.filemanager: Project dir monitoring is enabled for project "kf5-kdevplatform-devel (MP)"
    kdevplatform.shell:      "kf5-kdevplatform-devel (MP)" : creating dir watchers and/or starting parse jobs took an additional 1.921 seconds
    kdevplatform.filemanager: Project dir monitoring is enabled for project "kf5-kdevelop-5 (MP)"
    kdevplatform.shell: Project "kf5-kdevelop-5 (MP)" imported in 2.86 seconds
    kdevplatform.filemanager: Project dir monitoring is enabled for project "kf5-kdevelop-5 (MP)"
    kdevplatform.shell:      "kf5-kdevelop-5 (MP)" : creating dir watchers and/or starting parse jobs took an additional 2.86 seconds
  
  no dir watching, full parse (parsing cache uptodate)
  `deferred: import = 4—4.9 seconds, starting the parsers = 0.08—0.13 seconds`
  non deferred:
  
    kdevplatform.shell: Project "purpose-git" imported in 3.165 seconds
    kdevplatform.shell:      "purpose-git" : creating dir watchers and/or starting parse jobs took an additional 3.17 seconds
    kdevplatform.shell: Project "kf5-kdevplatform-devel (MP)" imported in 1.328 seconds
    kdevplatform.shell:      "kf5-kdevplatform-devel (MP)" : creating dir watchers and/or starting parse jobs took an additional 1.396 seconds
    kdevplatform.shell: Project "kf5-kdevelop-5 (MP)" imported in 1.517 seconds
    kdevplatform.shell:      "kf5-kdevelop-5 (MP)" : creating dir watchers and/or starting parse jobs took an additional 1.57 seconds
  
  2nd run
  
    kdevplatform.shell: Project "purpose-git" imported in 3.432 seconds
    kdevplatform.shell:      "purpose-git" : creating dir watchers and/or starting parse jobs took an additional 3.436 seconds
    kdevplatform.shell: Project "kf5-kdevplatform-devel (MP)" imported in 1.235 seconds
    kdevplatform.shell:      "kf5-kdevplatform-devel (MP)" : creating dir watchers and/or starting parse jobs took an additional 1.3 seconds
    kdevplatform.shell: Project "kf5-kdevelop-5 (MP)" imported in 1.598 seconds
    kdevplatform.shell:      "kf5-kdevelop-5 (MP)" : creating dir watchers and/or starting parse jobs took an additional 1.657 seconds
  
  Apparently the benefit of parser deferral is negligible on projects this size when the parser cache is uptodate and the system isn't stressed. Things hardly changed after I emptied the session cache directory:
  
    kdevplatform.shell: Project "purpose-git" imported in 3.382 seconds
    kdevplatform.shell:      "purpose-git" : creating dir watchers and/or starting parse jobs took an additional 3.387 seconds
    kdevplatform.shell: Project "kf5-kdevplatform-devel (MP)" imported in 1.603 seconds
    kdevplatform.shell:      "kf5-kdevplatform-devel (MP)" : creating dir watchers and/or starting parse jobs took an additional 1.633 seconds
    kdevplatform.shell: Project "kf5-kdevelop-5 (MP)" imported in 1.894 seconds
    kdevplatform.shell:      "kf5-kdevelop-5 (MP)" : creating dir watchers and/or starting parse jobs took an additional 1.971 seconds
  
  As I mentioned before, my initial idea for this change dates back to KDevelop4, where disabling "Parse All Project Files" did have a big effect on session load times (I also ran an earlier OS version without runtime RAM compression and with less RAM to boot so saw much more disk swapping).
  
  I continue to think though that the parser defer modification is small enough to be justified as it also makes things cleaner and should help keep the UI snappier when the system does become stressed.

REPOSITORY
  R32 KDevelop

REVISION DETAIL
  https://phabricator.kde.org/D7742

To: rjvbb, #kdevelop
Cc: kossebau, arrowdodger, brauch, zhigalin, kdevelop-devel, geetamc, Pilzschaf, akshaydeo, surgenight
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kdevelop-devel/attachments/20170909/3329873b/attachment-0001.html>