D7995: KDevelop: address dirwatching inefficiency (WIP/PoC)

[Aaron Puchert]

aaronpuchert added inline comments.

INLINE COMMENTS

> rjvbb wrote in projectwatcher.cpp:33
> No, that doesn't work properly across all platforms. Remember how I said KDirWatch uses a single QFileSystemWatcher instance to handle all KDirWatch instances using the QFSW method?
> 
> Because of that this code should never be executed in parallel, and I wanted to use a name that explains a bit better what the thing does than just `m_mutex`.
> 
> KDW could use a static mutex internally to protect its QFSW instance but we'd still be needing a mutex here so I'd propose to stick with a single, static external mutex for now. `KDW::addDir()` of a singe directory is cheap and fast compared to the other things going on, and chances that this method is being called concurrently are slim anyway so the potential performance penalty should be negligible.
> 
> As mentioned elsewhere: I've only seen the effects of not using a static mutex on Mac. Without mutex or with non-static mutex instances I get a significant increase in import times (variable but up to about 4x) and sometimes deadlocks occur (deep in the FSEvents backend to QFileSystemWatcher). This happens only when loading multiple projects that have a large number of almost empty directories, IOW, that call KDW::addDir() in rapid succession from multiple threads.
> 
> The only possible effect I've seen on Linux of not using a static mutex is an almost anecdotal double free deep in the KDW implementation. I haven't even been able to reproduce it but I'd never seen that happen before in all the years that I've been using KDevelop so I think that was some kind of race condition.

Calling a mutex `m_barrier` is a bit misleading: **mutex** comes from **mut**ual **ex**clusion. While that includes a //memory barrier// (preventing reordering around `lock` and `unlock`), it's more than that. Memory barriers don't guarantee that only one thread can access a resource at a given time.

On top of that, there is an actual synchronization mechanism called barrier <https://en.wikipedia.org/wiki/Barrier_(computer_science)>, which is something entirely different.

REPOSITORY
  R32 KDevelop

REVISION DETAIL
  https://phabricator.kde.org/D7995

To: rjvbb, #kdevelop, mwolff
Cc: aaronpuchert, arrowdodger, kfunk, dfaure, mwolff, brauch, kdevelop-devel, geetamc, Pilzschaf, akshaydeo, surgenight
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kdevelop-devel/attachments/20170927/05f3bef5/attachment.html>