D8579: KDevelop: project filter support in the abstractfilemanagerplugin import benchmark

René J.V. Bertin noreply at phabricator.kde.org
Wed Nov 1 16:44:31 UTC 2017


rjvbb added a comment.


  > is that really just the filter checks?
  
  Yes, I'd sure hope so, but you know the code (much) better than I do... The only difference between the 2 runs is that project plugins requiring GUI mode become available but from what I've seen they can only be used after calling `allPluginsForExtension()`. I only ask for project filter plugins in that call, and I haven't seen anything that suggests other plugins might be used as well by AbstractFileManagerPlugin.
  
  (On a side-note, if the author of plugin controller code is still active, s/he might want to consider adding some comments!)
  
  > Do you run your timings on a kdevelop codebase compiled with compiler optimizations? Most notably, is your Qt built with compiler optimizations?
  
  Yes and yes. KDevelop is built with clang 4.0.1 using `-Ofast -g` and Qt using GCC 6.2 with `-O3 -ftracer -g`.
  
  > My gut feeling is that the filters shouldn't be that bad, performance wise.
  
  That was my feeling too, until I realised that the filter is run on each and every file, meaning nearly 75k evaluations of I don't know how many regular expressions there are in the default filter set-up. Even if each filter costs about as much to apply to a project item as adding the item to the project you'd still end up multiplying the import duration by roughly the number of filters (in a serial implementation). Here we're talking about a factor 3.
  
  My Linux rig isn't exactly fast, either. Tests on my Mac show much less of an impact on the (6yo!) i7 powering it. Here I've shunted the call to KDirWatch::addDir() (which still takes almost 90 seconds and would pollute profiling measures).
  
    > abstractfilemanagerpluginimportbenchmark gcc-7.2.0
    The project filter plugin was not loaded
    KDirWatch backend: QFSWatch
    Starting import of project gcc-7.2.0
    	creating dirwatcher took 0 seconds
    importing 75039 items into project #0 took 2.401 seconds
    Done in 2.401 seconds total
        1.776 user_cpu 1.255 kernel_cpu 0:03.58 total_time 84.3%CPU
    The project filter plugin was not loaded
    KDirWatch backend: QFSWatch
    Starting import of project gcc-7.2.0
    	creating dirwatcher took 0 seconds
    importing 75039 items into project #0 took 2.401 seconds
    Done in 2.401 seconds total
        1.766 user_cpu 1.246 kernel_cpu 0:03.54 total_time 84.7%CPU
    The project filter plugin was not loaded
    KDirWatch backend: QFSWatch
    Starting import of project gcc-7.2.0
    	creating dirwatcher took 0 seconds
    importing 75039 items into project #0 took 2.406 seconds
    Done in 2.407 seconds total
        1.772 user_cpu 1.245 kernel_cpu 0:03.54 total_time 85.0%CPU
  
  
  
    > abstractfilemanagerpluginimportbenchmark --withFilters gcc-7.2.0
    cannot find .rc file "abstractfilemanagerpluginimportbenchmarkui.rc" for component "abstractfilemanagerpluginimportbenchmark"
    KDirWatch backend: QFSWatch
    Starting import of project gcc-7.2.0
    	creating dirwatcher took 0 seconds
    importing 74064 items into project #0 took 5.184 seconds
    Done in 5.187 seconds total
        4.584 user_cpu 1.476 kernel_cpu 0:07.96 total_time 76.0%CPU
    cannot find .rc file "abstractfilemanagerpluginimportbenchmarkui.rc" for component "abstractfilemanagerpluginimportbenchmark"
    KDirWatch backend: QFSWatch
    Starting import of project gcc-7.2.0
    	creating dirwatcher took 0 seconds
    importing 74064 items into project #0 took 5.129 seconds
    Done in 5.13 seconds total
        4.537 user_cpu 1.396 kernel_cpu 0:06.51 total_time 90.9%CPU
    cannot find .rc file "abstractfilemanagerpluginimportbenchmarkui.rc" for component "abstractfilemanagerpluginimportbenchmark"
    KDirWatch backend: QFSWatch
    Starting import of project gcc-7.2.0
    	creating dirwatcher took 0 seconds
    importing 74064 items into project #0 took 5.18 seconds
    Done in 5.18 seconds total
        4.560 user_cpu 1.400 kernel_cpu 0:06.51 total_time 91.5%CPU
  
  
  
  > Again, a profiler will tell you.
  
  Sure, but this is really an aspect I don't want to spend time on at this point (esp. since it turns out I don't even have to invoke the filter from my ProjectWatcher class). A quick run through Apple's Instruments suggests the overhead indeed comes from regular expression parsing (screenshot to follow).

REPOSITORY
  R32 KDevelop

REVISION DETAIL
  https://phabricator.kde.org/D8579

To: rjvbb, #kdevelop
Cc: mwolff, kdevelop-devel, njensen, geetamc, Pilzschaf, akshaydeo, surgenight, arrowdodger
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kdevelop-devel/attachments/20171101/c4ebba42/attachment.html>


More information about the KDevelop-devel mailing list