D7742: KDevelop projects: optional use of KDirWatch and defer it and parser start to after project loading

[René J.V. Bertin]

rjvbb added a comment.


  Some fresh timing results on Mac for importing an existing project for gcc 7.2.0 (full project parse off)
  
  Dirwatchers off:
  
    kdevplatform.shell: All projects imported in 8.324 seconds
    kdevplatform.shell:     creating dir watchers and/or starting parse jobs took an additional 0 seconds
  
  deferred dirwatcher creation:
  
    kdevplatform.shell: All projects imported in 7.846 seconds
    kdevplatform.filemanager: Project dir monitoring is enabled for project "gcc-7.2.0-MP"
    kdevplatform.shell:     creating dir watchers and/or starting parse jobs took an additional 85.419 seconds
    
    kdevplatform.shell: All projects imported in 8.207 seconds
    kdevplatform.filemanager: Project dir monitoring is enabled for project "gcc-7.2.0-MP"
    kdevplatform.shell:     creating dir watchers and/or starting parse jobs took an additional 85.039 seconds
  
  Those 85 seconds are spent "beachballing" because KDirWatch::addDir() blocks the event loop (a true busy wait with the CPU running at 100%).
  
  Overall process timing for that last run, when calling Quit (Command-Q) as soon as the dirwatcher creation started (= processed just after that completes):
  `246.719 user_cpu 13.199 kernel_cpu 4:25.29 total_time 97.9%CPU`
  
  In other words: destruction of the KDirWatch instance is probably even more expensive than the addDir() call. More precisely:
  
    kdevplatform.filemanager: Deleting dir watcher took 162.723  for project "gcc-7.2.0-MP"
    
    kdevplatform.shell: All projects imported in 7.937 seconds
    kdevplatform.filemanager: Project dir monitoring is enabled for project "gcc-7.2.0-MP"
    kdevplatform.shell:     creating dir watchers and/or starting parse jobs took an additional 84.48 seconds
    kdevplatform.filemanager: Deleting dir watcher took 160.892  for project "gcc-7.2.0-MP"
    245.998 user_cpu 13.800 kernel_cpu 4:23.85 total_time 98.4%CPU
  
  Those almost 3 minutes to close the gcc project are also spent outside of the event loop, and with the CPU running at 100%. There's a built-in way to defer KDirWatch destruction: using deleteLater() but even then it will still take place on (and thus blocking) the eventloop.
  
  Note to self: this NSLog message may be related to KDirWatch/QFSW not working properly on Mac:
  2017-09-19 14:55 kdevelop.bin[72054] (CarbonCore.framework) implementation_callback_rpc: ERROR: CFDictionaryGetValue(FSEvents_streamDict, cfPortNumber) => NULL

REPOSITORY
  R32 KDevelop

REVISION DETAIL
  https://phabricator.kde.org/D7742

To: rjvbb, #kdevelop, mwolff, brauch
Cc: mwolff, kossebau, arrowdodger, brauch, zhigalin, kdevelop-devel, geetamc, Pilzschaf, akshaydeo, surgenight
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kdevelop-devel/attachments/20170919/c4ccb4cb/attachment.html>