<table><tr><td style="">rjvbb added a comment.
</td><a style="text-decoration: none; padding: 4px 8px; margin: 0 8px 8px; float: right; color: #464C5C; font-weight: bold; border-radius: 3px; background-color: #F7F7F9; background-image: linear-gradient(to bottom,#fff,#f1f0f1); display: inline-block; border: 1px solid rgba(71,87,120,.2);" href="https://phabricator.kde.org/D8579" rel="noreferrer">View Revision</a></tr></table><br /><div><div><blockquote style="border-left: 3px solid #a7b5bf; color: #464c5c; font-style: italic; margin: 4px 0 12px 0; padding: 4px 12px; background-color: #f8f9fc;"><p>is that really just the filter checks?</p></blockquote>
<p>Yes, I'd sure hope so, but you know the code (much) better than I do... The only difference between the 2 runs is that project plugins requiring GUI mode become available but from what I've seen they can only be used after calling <tt style="background: #ebebeb; font-size: 13px;">allPluginsForExtension()</tt>. I only ask for project filter plugins in that call, and I haven't seen anything that suggests other plugins might be used as well by AbstractFileManagerPlugin.</p>
<p>(On a side-note, if the author of plugin controller code is still active, s/he might want to consider adding some comments!)</p>
<blockquote style="border-left: 3px solid #a7b5bf; color: #464c5c; font-style: italic; margin: 4px 0 12px 0; padding: 4px 12px; background-color: #f8f9fc;"><p>Do you run your timings on a kdevelop codebase compiled with compiler optimizations? Most notably, is your Qt built with compiler optimizations?</p></blockquote>
<p>Yes and yes. KDevelop is built with clang 4.0.1 using <tt style="background: #ebebeb; font-size: 13px;">-Ofast -g</tt> and Qt using GCC 6.2 with <tt style="background: #ebebeb; font-size: 13px;">-O3 -ftracer -g</tt>.</p>
<blockquote style="border-left: 3px solid #a7b5bf; color: #464c5c; font-style: italic; margin: 4px 0 12px 0; padding: 4px 12px; background-color: #f8f9fc;"><p>My gut feeling is that the filters shouldn't be that bad, performance wise.</p></blockquote>
<p>That was my feeling too, until I realised that the filter is run on each and every file, meaning nearly 75k evaluations of I don't know how many regular expressions there are in the default filter set-up. Even if each filter costs about as much to apply to a project item as adding the item to the project you'd still end up multiplying the import duration by roughly the number of filters (in a serial implementation). Here we're talking about a factor 3.</p>
<p>My Linux rig isn't exactly fast, either. Tests on my Mac show much less of an impact on the (6yo!) i7 powering it. Here I've shunted the call to KDirWatch::addDir() (which still takes almost 90 seconds and would pollute profiling measures).</p>
<div class="remarkup-code-block" style="margin: 12px 0;" data-code-lang="text" data-sigil="remarkup-code-block"><pre class="remarkup-code" style="font: 11px/15px "Menlo", "Consolas", "Monaco", monospace; padding: 12px; margin: 0; background: rgba(71, 87, 120, 0.08);">> abstractfilemanagerpluginimportbenchmark gcc-7.2.0
The project filter plugin was not loaded
KDirWatch backend: QFSWatch
Starting import of project gcc-7.2.0
creating dirwatcher took 0 seconds
importing 75039 items into project #0 took 2.401 seconds
Done in 2.401 seconds total
1.776 user_cpu 1.255 kernel_cpu 0:03.58 total_time 84.3%CPU
The project filter plugin was not loaded
KDirWatch backend: QFSWatch
Starting import of project gcc-7.2.0
creating dirwatcher took 0 seconds
importing 75039 items into project #0 took 2.401 seconds
Done in 2.401 seconds total
1.766 user_cpu 1.246 kernel_cpu 0:03.54 total_time 84.7%CPU
The project filter plugin was not loaded
KDirWatch backend: QFSWatch
Starting import of project gcc-7.2.0
creating dirwatcher took 0 seconds
importing 75039 items into project #0 took 2.406 seconds
Done in 2.407 seconds total
1.772 user_cpu 1.245 kernel_cpu 0:03.54 total_time 85.0%CPU</pre></div>
<div class="remarkup-code-block" style="margin: 12px 0;" data-code-lang="text" data-sigil="remarkup-code-block"><pre class="remarkup-code" style="font: 11px/15px "Menlo", "Consolas", "Monaco", monospace; padding: 12px; margin: 0; background: rgba(71, 87, 120, 0.08);">> abstractfilemanagerpluginimportbenchmark --withFilters gcc-7.2.0
cannot find .rc file "abstractfilemanagerpluginimportbenchmarkui.rc" for component "abstractfilemanagerpluginimportbenchmark"
KDirWatch backend: QFSWatch
Starting import of project gcc-7.2.0
creating dirwatcher took 0 seconds
importing 74064 items into project #0 took 5.184 seconds
Done in 5.187 seconds total
4.584 user_cpu 1.476 kernel_cpu 0:07.96 total_time 76.0%CPU
cannot find .rc file "abstractfilemanagerpluginimportbenchmarkui.rc" for component "abstractfilemanagerpluginimportbenchmark"
KDirWatch backend: QFSWatch
Starting import of project gcc-7.2.0
creating dirwatcher took 0 seconds
importing 74064 items into project #0 took 5.129 seconds
Done in 5.13 seconds total
4.537 user_cpu 1.396 kernel_cpu 0:06.51 total_time 90.9%CPU
cannot find .rc file "abstractfilemanagerpluginimportbenchmarkui.rc" for component "abstractfilemanagerpluginimportbenchmark"
KDirWatch backend: QFSWatch
Starting import of project gcc-7.2.0
creating dirwatcher took 0 seconds
importing 74064 items into project #0 took 5.18 seconds
Done in 5.18 seconds total
4.560 user_cpu 1.400 kernel_cpu 0:06.51 total_time 91.5%CPU</pre></div>
<blockquote style="border-left: 3px solid #a7b5bf; color: #464c5c; font-style: italic; margin: 4px 0 12px 0; padding: 4px 12px; background-color: #f8f9fc;"><p>Again, a profiler will tell you.</p></blockquote>
<p>Sure, but this is really an aspect I don't want to spend time on at this point (esp. since it turns out I don't even have to invoke the filter from my ProjectWatcher class). A quick run through Apple's Instruments suggests the overhead indeed comes from regular expression parsing (screenshot to follow).</p></div></div><br /><div><strong>REPOSITORY</strong><div><div>R32 KDevelop</div></div></div><br /><div><strong>REVISION DETAIL</strong><div><a href="https://phabricator.kde.org/D8579" rel="noreferrer">https://phabricator.kde.org/D8579</a></div></div><br /><div><strong>To: </strong>rjvbb, KDevelop<br /><strong>Cc: </strong>mwolff, kdevelop-devel, njensen, geetamc, Pilzschaf, akshaydeo, surgenight, arrowdodger<br /></div>