D7995: KDevelop: address dirwatching inefficiency (WIP/PoC)

[René J.V. Bertin]

rjvbb added inline comments.

INLINE COMMENTS

> mwolff wrote in abstractfilemanagerplugin.cpp:126
> remove the conditionals, always output the time?

fine with me, someone suggested I add the conditional ;)

> mwolff wrote in abstractfilemanagerplugin.h:106
> actually, isn't this now unsafe API? If the dir watcher needs to be protected by a mutex, we must not hand it out as a raw pointer, no?

I don't think it's any more unsafe than it already was; I don't see anything that suggests that using a KDirWatch once from another thread makes it unsafe to use from its main thread afterwards. Or rather, concurrently from multiple threads because that's what the mutex protects against.
Maybe David can tell us otherwise, but a priori it should be enough to add a warning in the comment that KDirWatch isn't thread-safe.

I've tried my benchmark app without the mutex locally, and managed to reproduce the double-free crash quite regularly (on Linux). I think it would have been observed in the wild by now if the result from `projectWatcher` was used concurrently.

> mwolff wrote in abstractfilemanagerplugin.h:111
> this should be moved into KDirWatch, then we can use the getter above

I agree it's lacking from KDirWatch, but can't we develop the change here first and then see what needs to be upstreamed?

> mwolff wrote in abstractfilemanagerplugin.h:114
> so it's like closing the project, no? why do

Why do what?

It's part of what happens when closing a project in KDevelop, but it isn't actually closing the project. It's telling the file manager to forget about the project. There is no current use case for the method in KDevelop itself, it's only for the benchmark. I tried to get the signal sent through the ProjectController (`projectController()->closeProject(m_project)`) but that seems to need more setting up. I don't have time to figure out what and if that is going to add noise to the benchmark (parser overhead, for instance).

The `detach` method could be useful in a mode where you want to keep a project open for quick reference (or snatching some code snippets) only and don't want to waste resources on parsing etc.

> mwolff wrote in filemanagerlistjob.cpp:108
> aren't you adding the same dir twice now? i.e. once here, and once above when the dir gets listed itself?

Did I misunderstand, doesn't this loop add the contents of the folder at `path`?

> mwolff wrote in abstractfilemanagerpluginimportbenchmark.cpp:48
> are you only changing visibility here? if so, use `using` instead, or better yet - add the `AFMPBenchmark` as friend class to the `AFMP`

Yes, just visibility change. I didn't think it was OK to add a (private) test class as a friend to public API. Will do that, then.

> mwolff wrote in abstractfilemanagerpluginimportbenchmark.cpp:138
> use a plain vector or std::array or c-array? Or are the enumerators non-contiguous? if the latter, use qhash

The enumerator doesn't specify values; is there a standard that guarantees that such an enum always counts from 0 by 1?

> mwolff wrote in abstractfilemanagerpluginimportbenchmark.cpp:158
> I think you misunderstood me. I don't want to keep the benchmark of the old implementation around. What I want is a simple benchmark (separate patch!) that can be run on the current implementation. Then when one applies your optimization patch, it should show a performance improvement of the benchmark.

Well, you didn't clarify that so I did what cost me the least time (for the initial implementation). I'm ok with adding a benchmark first as long as that can also go into the 5.2 branch, along with any API additions like `AbstractFileManagerPlugin::detach()` (or however we'll be calling it).

I was kind of hoping that everyone who actually uses the benchmark would see the obvious performance improvement when using `KDIRWATCH_METHOD=QFSWatch` during this review process, which would remove the need to commit the BENCHMARK_ORIGINAL_DIRWATCHER feature.

It *was* the idea to use the benchmark before committing it, no?

> mwolff wrote in abstractfilemanagerpluginimportbenchmark.cpp:172
> range-based for

Sorry, what's the difference?

REPOSITORY
  R32 KDevelop

REVISION DETAIL
  https://phabricator.kde.org/D7995

To: rjvbb, #kdevelop, mwolff
Cc: aaronpuchert, arrowdodger, kfunk, dfaure, mwolff, brauch, kdevelop-devel, geetamc, Pilzschaf, akshaydeo, surgenight
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kdevelop-devel/attachments/20170928/334ce617/attachment.html>