KDevelop can't load project: "too large document"?

[Matthew Woehlke]

On 2017-12-12 13:52, René J.V. Bertin wrote:
> On Tuesday December 12 2017 11:50:38 Matthew Woehlke wrote:
>>> That'd be the ultimate test, but I'm also not seeing similar sizes
>>> with other projects like KDevelop and QtWebKit.
>> ...and the analysis I posted yesterday shows why; KWIVER has an
>> unusually high ratio of related statements, and a LOT of targets; more
> 
> That's not the reason why you get larger values for those projects than I do.

Sigh. I sent a reply to that that's been stuck in moderation.

You mentioned a KDevelop code model that is "0.004Mb", or about 4 KiB.
This seems... implausible, especially if it has backtraces. Either you
have enormously fewer targets than I do for some reason, or you aren't
looking at the (full) code model.

As an experiment, please feed the attached file as stdin to `cmake -E
server --debug --experimental` (modifying the paths in it to reflect
your KDevelop source/build) and look at the size of the output. (Note:
redirect it *all* to a file; trying to directly process it seems to
always truncate the output for me. You can feed the resulting output
file to `sed 'x;$!d' | wc -c` to get the code model size.)

If you're *really* getting a code model on the order of a few KiB, I
would be extremely curious to see what's in it...

(In the message that's stuck in moderation, I also included my own code
model, which is why it's waiting for moderator approval; it's 45K
XZ-compressed. I don't recall if that's the version with or without
backtraces. Here I am attaching just the target list; I skimmed it and
nothing seems unusual, and it does not contain any duplicates.)

>> What enormous files are being written to disk? My build.ninja is only
>> 637 KiB...
> 
> My test-project/build directory is 49Mb and that's with lz4 
> compression. This is concentrated in the CMakeFiles directories
> which has 1 .dir folder per target (and I'm a bit at a loss
> determining whether the compression doesn't actually increase the
> effective size here).
Ah. *Wow*, same here; I hadn't thought to look at CMakeFiles. Okay, I
can think of many plausible reasons your *I/O* would be slower, so that
explains things. Thanks.

(I encourage you to try Ninja; besides being a very nice build tool in
general, the CMakeFiles for Ninja for that project is only 720 KiB ;-).
Also FWIW, `du -sh` shows me 46M, so yeah, compression is probably not
helping the size significantly.)

-- 
Matthew
-------------- next part --------------
[== "CMake Server" ==[
{
    "buildDirectory": "/usr/local/build/matthew/kde/kdevelop",
    "protocolVersion": {
        "major": 1
    },
    "sourceDirectory": "/usr/local/src/matthew/kde/kdevelop",
    "type": "handshake"
}
]== "CMake Server" ==]


[== "CMake Server" ==[
{
  "cacheArguments": [],
  "type": "configure"
}
]== "CMake Server" ==]

[== "CMake Server" ==[
{
  "type": "compute"
}
]== "CMake Server" ==]

[== "CMake Server" ==[
{
  "type": "codemodel"
}
]== "CMake Server" ==]
-------------- next part --------------
ExperimentalBuild
ContinuousUpdate
ExperimentalStart
Nightly
ContinuousStart
ExperimentalCoverage
NightlyConfigure
NightlyCoverage
tsfiles
NightlyTest
pofiles
Continuous
ContinuousBuild
ExperimentalUpdate
ContinuousTest
NightlySubmit
NightlyBuild
NightlyMemoryCheck
NightlyStart
NightlyUpdate
ExperimentalTest
NightlyMemCheck
ExperimentalSubmit
ContinuousConfigure
ContinuousMemCheck
ExperimentalConfigure
ContinuousSubmit
uninstall
ExperimentalMemCheck
fetch-translations
Experimental
pofiles-f83bb36daf8869facfa5a1446fcf9767
ContinuousCoverage
tsfiles-f83bb36daf8869facfa5a1446fcf9767
KDevPlatformSublime
example2
example1
test_toolviewtoolbar
test_areawalker
test_controller
test_areaoperation
test_view
test_document
test_aggregatemodel
test_viewactivation
KDevPlatformInterfaces
KDevPlatformProject
projectmodelperformancetest
test_projectmodel
abstractfilemanagerpluginimporttest
abstractfilemanagerpluginimportbenchmark
KDevPlatformLanguage
test_highlighting
test_identifier
test_duchain
test_duchainshutdown
test_stringhelpers
bench_hashes
test_backgroundparser
test_templaterenderer
test_documentchangeset
test_templateclassgenerator
test_templatesmodel
test_cpp
test_yaml
test_options
test_kdevhash
KDevPlatformShell
test_checkerstatus
test_filteredproblemstore
test_problemstore
test_problemstorenode
test_problemmodelset
test_sessioncontroller
test_shellbuddy
test_shelldocumentoperation
projectdefaultplugin
test_problemmodel
test_uicontroller
globalnondefaultplugin
nonguiinterfaceplugin
test_projectcontroller
test_documentcontroller
globaldefaultplugin
test_pluginenabling
test_plugincontroller
test_testcontroller
projectnondefaultplugin
test_ktexteditorpluginintegration
test_detectedproblem
kdev_format_source
KDevPlatformUtil
kdev_dbus_socket_transformer
duchainify
test_texteditorhelpers
test_kdevformatsource
test_path
test_embeddedfreetree
test_kdevvarlengtharray
test_foregroundlock
test_executecompositejob
test_stringhandler
test_objectlist
test_environment
KDevPlatformOutputView
test_filteringstrategy
test_outputmodel
KDevPlatformVcs
test_vcslocation
test_vcsannotationline
test_vcsstatusinfo
test_vcsrevision
test_vcsitemevent
test_vcsdiff
test_vcsevent
test_vcsannotation
test_dvcsjob
test_models
KDevPlatformDebugger
test_ivariablecontroller
KDevPlatformDocumentation
KDevPlatformSerialization
test_itemrepository
bench_itemrepository
test_itemrepositoryregistry_automatic
test_itemrepositoryregistry_deferred
test_indexedstring
kdev_filters
KDevPlatformTests
kdevcppcheck
kdevcppcheck_core
test_cppcheckjob
test_cppcheckparser
kdevheaptrack
kdevheaptrack_config
kdevdebuggercommon
kdevdbg_testhelper
debuggee_qlistcontainer
debuggee_qstring
debuggee_debugeeexception
debuggee_qhashstring
debuggee_qbytearray
debuggee_debugeethreads
debuggee_debugeepath
debuggee_debugeemultiplebreakpoint
debuggee_debugeeechoenv
debuggee_quuid
debuggee_qmapint
debuggee_crash
debuggee_debugeerecursion
debuggee_qdatetime
debuggee_qtime
debuggee_debugeeslow
debuggee_debugeemultilocbreakpoint
debuggee_qmapstring
debuggee_debugee
debuggee_qmapstringbool
debuggee_qdate
debuggee_qsetint
debuggee_qchar
cpp_basic
debuggee_ktexteditortypes
debuggee_qurl
debuggee_qhashint
debuggee_debugeeqt
debuggee_qsetstring
debuggee_kdeveloptypes
debuggee_qlistpod
debuggee_debugeespace
debuggee_spacedebugee
kdevlldb
kdevlldb_static
test_lldb
test_lldbformatters
kdevgdb
kdevgdb_static
test_gdbprinters
test_gdb
kdevqthelp
test_qthelpplugin
kdevmanpage
test_manpagemodel
kdevastyle
astylelib
test_astyle
kdevcustomscript
kdev_includepathsconverter
kdevdefinesandincludesmanager
kdevdefinesandincludesmanager_interface
test_definesandincludes
kdevcompilerprovider
test_compilerprovider
kdevnoprojectincludesanddefines
kdevqmljslanguagesupport
kdevqtc-qmlsupport
kdevqmljsduchain
test_qmljsdeclarations
test_qmljscontexts
test_files
qml-parser
kdevqmljscompletion
test_qmljscompletion
KDevClangPrivate
kdevclangsupport
test_assistants
test_buddies-clang
test_codecompletion
test_files-clang
clang-minimal-visitor
clang-parser
test_clangutils
test_duchain-clang
test_refactoring-clang
codecompletiontestbase
test_duchainutils
test_problems
bench_codecompletion
KDevIMakeBuilder
kdevmakebuilder
kdevninja
kdevcmakebuilder
kdevqmakebuilder_interface
kdevqmakebuilder
kdevcmakemanagernosettings
kdevcmakecommon
kdevcmakemanager
kdevcmakedocumentation
test_ctestfindsuites
test_cmakeparser
test_cmakeserver
test_cmakemanager
kdevprojectopen
kdevcustommakemanager
kdevmakefileresolver
test_custommake
kdevcustombuildsystem
kcm_uitest
test_custombuildsystemplugin
kdevqmakemanager
kdevqmakecommon
kdevqmakeparser
test_qmakeassignment
test_qmakelexer
test_qmakeparse
test_qmakefunctionscope
test_qmakescope
test_qmakeproject
test_qmakefile
kdevgenericmanager
test_projectload
kdevandroid
kdevdocker
dockerfile-template
test_docker
kdevflatpak
flatpak-template
kdevbazaar
kdevcvs
kdevgit
test_kdevgit
kdevperforce
p4clientstub
test_kdevperforce
kdevsubversion
test_svnrecursiveadd
test_svnimport
kdevsvncpp
kdevvcschangesviewplugin
kdevappwizard
kdevcodeutils
kdevcontextbrowser
kdevdocumentswitcher
kdevdocumentview
kdevexecute
KDevPlatformIExecute
kdevexecutescript
kdevexternalscript
kdevfilemanager
testfiletemplates
kdevfiletemplates
test_generationtest
test_yaml2
test_qtestlib
kdevgrepview
test_findreplace
kdevopenwith
kdevoutlineview
kdevpatchreview
kdevproblemreporter
test_problemsview
kdevprojectfilter
test_projectfilter
kdevprojectmanagerview
kdevquickopen
test_quickopen
quickopentestbase
bench_quickopen
kdevsourceformatter
kdevstandardoutputview
test_standardoutputview
kdevswitchtobuddy
kdevtestview
kdevclassbrowser
kdevexecuteplasmoid
kdevghprovider
kdevkdeprovider
test_kdeprojectsreader
kdevkonsoleview
kdevwelcomepage
kdevelopdashboarddeclarativeplugin
kdevokteta
kdevelop
plasma_engine_kdevelopsessions
krunner_kdevelopsessions
empty
cmake_plaincpp
cmake_qt5guiapp
cmake_kdevplugin
qt5-qml2
cmake_qt5-qml2
qmake_qt5guiapp
php_phpunit
python_pyunit
cpp_qtestlib_kdevelop
cpp_gtest
cpp_cpputest
c_gobject_properties
qobject_pimpl
c_gobject_private
c_gobject
python_basic
private_pointer
qt_interface
qt_shared
qabstractitemmodel
qobject
cmake_module
qabstractitemmodel_pimpl
qdialog_pimpl
qdialog
qt_widget
cpp_qtestlib
qwidget_pimpl
doc-kdevelop-index-cache-bz2
KDevPlatformSublime_autogen
example2_autogen
example1_autogen
test_toolviewtoolbar_autogen
test_areawalker_autogen
test_controller_autogen
test_areaoperation_autogen
test_view_autogen
test_document_autogen
test_aggregatemodel_autogen
test_viewactivation_autogen
KDevPlatformInterfaces_autogen
KDevPlatformProject_autogen
projectmodelperformancetest_autogen
test_projectmodel_autogen
abstractfilemanagerpluginimporttest_autogen
abstractfilemanagerpluginimportbenchmark_autogen
test_highlighting_autogen
KDevPlatformLanguage_autogen
test_identifier_autogen
test_duchain_autogen
test_duchainshutdown_autogen
test_stringhelpers_autogen
bench_hashes_autogen
test_backgroundparser_autogen
test_templaterenderer_autogen
test_documentchangeset_autogen
test_templateclassgenerator_autogen
test_templatesmodel_autogen
test_kdevhash_autogen
test_checkerstatus_autogen
KDevPlatformShell_autogen
test_filteredproblemstore_autogen
test_problemstore_autogen
test_problemstorenode_autogen
test_problemmodelset_autogen
test_sessioncontroller_autogen
test_shellbuddy_autogen
test_shelldocumentoperation_autogen
projectdefaultplugin_autogen
test_problemmodel_autogen
test_uicontroller_autogen
globalnondefaultplugin_autogen
nonguiinterfaceplugin_autogen
test_projectcontroller_autogen
test_documentcontroller_autogen
globaldefaultplugin_autogen
test_pluginenabling_autogen
test_plugincontroller_autogen
test_testcontroller_autogen
projectnondefaultplugin_autogen
test_ktexteditorpluginintegration_autogen
test_detectedproblem_autogen
kdev_format_source_autogen
KDevPlatformUtil_autogen
kdev_dbus_socket_transformer_autogen
duchainify_autogen
test_texteditorhelpers_autogen
test_kdevformatsource_autogen
test_path_autogen
test_executecompositejob_autogen
test_embeddedfreetree_autogen
test_kdevvarlengtharray_autogen
test_foregroundlock_autogen
test_objectlist_autogen
test_stringhandler_autogen
test_environment_autogen
KDevPlatformOutputView_autogen
test_filteringstrategy_autogen
KDevPlatformVcs_autogen
test_outputmodel_autogen
test_vcslocation_autogen
test_vcsannotationline_autogen
test_vcsstatusinfo_autogen
test_vcsrevision_autogen
test_vcsitemevent_autogen
test_vcsdiff_autogen
test_vcsevent_autogen
test_vcsannotation_autogen
test_dvcsjob_autogen
test_models_autogen
test_ivariablecontroller_autogen
KDevPlatformDebugger_autogen
KDevPlatformDocumentation_autogen
KDevPlatformSerialization_autogen
test_itemrepository_autogen
bench_itemrepository_autogen
test_itemrepositoryregistry_automatic_autogen
test_itemrepositoryregistry_deferred_autogen
test_indexedstring_autogen
kdev_filters_autogen
KDevPlatformTests_autogen
kdevcppcheck_autogen
kdevcppcheck_core_autogen
test_cppcheckjob_autogen
test_cppcheckparser_autogen
kdevheaptrack_autogen
kdevheaptrack_config_autogen
kdevdbg_testhelper_autogen
kdevdebuggercommon_autogen
debuggee_qlistcontainer_autogen
debuggee_qstring_autogen
debuggee_debugeeexception_autogen
debuggee_qhashstring_autogen
debuggee_qbytearray_autogen
debuggee_debugeethreads_autogen
debuggee_debugeepath_autogen
debuggee_debugeemultiplebreakpoint_autogen
debuggee_debugeeechoenv_autogen
debuggee_quuid_autogen
debuggee_qmapint_autogen
debuggee_crash_autogen
debuggee_debugeerecursion_autogen
debuggee_qdatetime_autogen
debuggee_qtime_autogen
debuggee_debugeeslow_autogen
debuggee_debugeemultilocbreakpoint_autogen
debuggee_qmapstring_autogen
debuggee_debugee_autogen
debuggee_qmapstringbool_autogen
debuggee_qdate_autogen
debuggee_qsetint_autogen
debuggee_qchar_autogen
debuggee_ktexteditortypes_autogen
debuggee_qurl_autogen
debuggee_qhashint_autogen
debuggee_debugeeqt_autogen
debuggee_qsetstring_autogen
debuggee_kdeveloptypes_autogen
debuggee_qlistpod_autogen
debuggee_debugeespace_autogen
debuggee_spacedebugee_autogen
kdevlldb_autogen
kdevlldb_static_autogen
test_lldb_autogen
test_lldbformatters_autogen
kdevgdb_autogen
test_gdbprinters_autogen
kdevgdb_static_autogen
kdevqthelp_autogen
test_gdb_autogen
test_qthelpplugin_autogen
kdevmanpage_autogen
test_manpagemodel_autogen
kdevastyle_autogen
astylelib_autogen
test_astyle_autogen
kdevcustomscript_autogen
kdev_includepathsconverter_autogen
kdevdefinesandincludesmanager_autogen
test_definesandincludes_autogen
kdevcompilerprovider_autogen
test_compilerprovider_autogen
kdevnoprojectincludesanddefines_autogen
kdevqmljslanguagesupport_autogen
kdevqtc-qmlsupport_autogen
kdevqmljsduchain_autogen
test_qmljsdeclarations_autogen
test_qmljscontexts_autogen
test_files_autogen
qml-parser_autogen
kdevqmljscompletion_autogen
test_qmljscompletion_autogen
KDevClangPrivate_autogen
kdevclangsupport_autogen
test_assistants_autogen
test_buddies-clang_autogen
test_codecompletion_autogen
test_files-clang_autogen
clang-minimal-visitor_autogen
clang-parser_autogen
test_clangutils_autogen
test_duchain-clang_autogen
test_refactoring-clang_autogen
codecompletiontestbase_autogen
test_duchainutils_autogen
test_problems_autogen
bench_codecompletion_autogen
kdevmakebuilder_autogen
kdevninja_autogen
kdevcmakebuilder_autogen
kdevqmakebuilder_autogen
kdevcmakemanagernosettings_autogen
kdevcmakecommon_autogen
kdevcmakemanager_autogen
kdevcmakedocumentation_autogen
test_ctestfindsuites_autogen
test_cmakeparser_autogen
test_cmakeserver_autogen
test_cmakemanager_autogen
kdevprojectopen_autogen
kdevcustommakemanager_autogen
kdevmakefileresolver_autogen
test_custommake_autogen
kdevcustombuildsystem_autogen
kcm_uitest_autogen
test_custombuildsystemplugin_autogen
kdevqmakemanager_autogen
kdevqmakecommon_autogen
kdevqmakeparser_autogen
test_qmakeassignment_autogen
test_qmakelexer_autogen
test_qmakeparse_autogen
test_qmakefunctionscope_autogen
test_qmakescope_autogen
test_qmakeproject_autogen
test_qmakefile_autogen
kdevgenericmanager_autogen
test_projectload_autogen
kdevandroid_autogen
kdevdocker_autogen
test_docker_autogen
kdevflatpak_autogen
kdevbazaar_autogen
kdevcvs_autogen
kdevgit_autogen
test_kdevgit_autogen
kdevperforce_autogen
p4clientstub_autogen
test_kdevperforce_autogen
kdevsubversion_autogen
test_svnrecursiveadd_autogen
test_svnimport_autogen
kdevsvncpp_autogen
kdevvcschangesviewplugin_autogen
kdevappwizard_autogen
kdevcodeutils_autogen
kdevcontextbrowser_autogen
kdevdocumentswitcher_autogen
kdevdocumentview_autogen
kdevexecute_autogen
kdevexecutescript_autogen
kdevexternalscript_autogen
kdevfilemanager_autogen
testfiletemplates_autogen
kdevfiletemplates_autogen
test_generationtest_autogen
kdevgrepview_autogen
test_findreplace_autogen
kdevopenwith_autogen
kdevpatchreview_autogen
kdevoutlineview_autogen
test_problemsview_autogen
kdevproblemreporter_autogen
kdevprojectfilter_autogen
test_projectfilter_autogen
kdevprojectmanagerview_autogen
kdevquickopen_autogen
test_quickopen_autogen
quickopentestbase_autogen
kdevsourceformatter_autogen
bench_quickopen_autogen
kdevstandardoutputview_autogen
test_standardoutputview_autogen
kdevswitchtobuddy_autogen
kdevtestview_autogen
kdevclassbrowser_autogen
kdevghprovider_autogen
kdevexecuteplasmoid_autogen
kdevkdeprovider_autogen
test_kdeprojectsreader_autogen
kdevkonsoleview_autogen
kdevwelcomepage_autogen
kdevelopdashboarddeclarativeplugin_autogen
kdevokteta_autogen
kdevelop_autogen
plasma_engine_kdevelopsessions_autogen
krunner_kdevelopsessions_autogen