Very slow background parsing

[Dmitry Risenberg]

Hi all!

Recently the background parser has become very slow on my machine - 30
minutes after startup it is only about 40% finished. I wasn't able to
find the reason for that - rolling back to an older repository
snapshot did not fix it, and no relevent updates to the system itself
were installed. Besides, this affected only one of my two PCs.
This also doesn't seem to be the bug with zlib version - I have
"1.2.3" as version.

The only clue about such behaviour that I did find are entries in the
debug output like the following one:

kdevelop(23360)/kdevelop (cpp support) PreprocessJob::sourceNeeded:
PreprocessJob "/home/drisenberg/Development/KDevelop/kdevelop/build/languages/cpp/ui_custom_include_paths.h"
: need to update KUrl("file:///usr/include/qt4/QtGui/QHeaderView")
kdevelop(23360)/kdevelop (cpp support) PreprocessJob::run:
PreprocessJob: preprocessing "/usr/include/qt4/QtGui/QHeaderView"
kdevelop(23360)/kdevelop (cpp support) CPPInternalParseJob::run: ===--
PARSING --===>  "/usr/include/KDE/KGlobal"
kdevelop(23360)/kdevelop (cpp support) CPPInternalParseJob::run:
building duchain for "/usr/include/KDE/KGlobal"
kdevelop(23360)/kdevelop (cpp support) ContextBuilder::buildContexts:
ContextBuilder::buildContexts: compiling
kdevelop(23360)/kdevelop (cpp support) ContextBuilder::buildContexts:
built top-level context with 0 declarations and 9 included files
kdevelop(23360)/kdevelop (cpp support)
ContextBuilder::buildProxyContextFromContent:
ContextBuilder::buildProxyContextFromContent: compiling
kdevelop(23360)/kdevelop (cpp support) CPPInternalParseJob::run: ===--
Parsing finished --===> "/usr/include/KDE/KGlobal"
kdevelop(23360)/kdevplatform (language)
KDevelop::DUChainPrivate::loadChain: loading top-context 2293
kdevelop(23360)/kdevplatform (language)
KDevelop::DUChainPrivate::loadChain: loading top-context 1597
kdevelop(23360)/kdevelop (cpp support) PreprocessJob::sourceNeeded:
PreprocessJob "/usr/include/qt4/QtGui/QHeaderView" : need to update
KUrl("file:///usr/include/qt4/QtGui/qheaderview.h")

There are multiple entries with such pattern: a Qt-generated header
file calls reparsing of a Qt header, which in turn reparses its own
includes, and so on. This is strange, because
ui_custom_include_paths.h doesn't define any macros before the
includes, except for the guard, so there's no need for an update.

I also tried running duchainify on kdevelop project directory with
--force-update-recursive, which gave much better time - 11 minutes. In
the log there were the following entries:
duchainify(18660)/kdevelop (cpp support) CppUtils::findInclude: FAILED
to find include-file "QtGui/QHeaderView" in paths:
duchainify(18660)/kdevelop (cpp support) CppUtils::findInclude:
KUrl("file:///usr/include/c++/4.4")
duchainify(18660)/kdevelop (cpp support) CppUtils::findInclude:
KUrl("file:///usr/include/c++/4.4/x86_64-linux-gnu")
duchainify(18660)/kdevelop (cpp support) CppUtils::findInclude:
KUrl("file:///usr/include/c++/4.4/backward")
duchainify(18660)/kdevelop (cpp support) CppUtils::findInclude:
KUrl("file:///usr/local/include")
duchainify(18660)/kdevelop (cpp support) CppUtils::findInclude:
KUrl("file:///usr/lib/gcc/x86_64-linux-gnu/4.4.5/include")
duchainify(18660)/kdevelop (cpp support) CppUtils::findInclude:
KUrl("file:///usr/lib/gcc/x86_64-linux-gnu/4.4.5/include-fixed")
duchainify(18660)/kdevelop (cpp support) CppUtils::findInclude:
KUrl("file:///usr/include")
duchainify(18660)/kdevelop (cpp support) PreprocessJob::sourceNeeded:
PreprocessJob "../../kdevelop/build/languages/cpp/ui_custom_include_paths.h"
: include not found: "QtGui/QHeaderView"

So may be duchainify was showing a better time because it just failed
to do the full re-parsing.

I'm not attaching the full parsing logs here, because they are quite
big (2.5 Mb zipped), but can attach them if necessary.

-- 
Dmitry Risenberg