KDevelop can't load project: "too large document"?

Matthew Woehlke mwoehlke.floss at gmail.com
Mon Dec 11 22:11:14 GMT 2017


(Resend without the huge attachment; if a moderator is reading this,
please DO NOT approve the version with the 300 KiB attachment; thanks!
...and sorry 'bout that.)

On 2017-12-11 12:26, Matthew Woehlke wrote:
> This does suggest there is something a bit pathological about KWIVER
> w.r.t. backtraces...

This seemed odd, so I did a little more digging, with the help of a
quick tool I whipped together to compute JSON statistics (basically, it
records paths to values and counts how many values have a particular path).

>From this, I was able to determine:

- KDevelop has 612 CMake targets producing 268 artifacts with an average
of 2.40 lines of cross-reference backtrace and an average of 9.45
related statements per target, which in turn have an average of 2.48
lines of backtrace.

- KWIVER has 7700¹ CMake targets producing 390 artifacts with an average
of 5.02 lines of cross-reference backtrace and an average of 28.84
related statements per target, which in turn have an average of 4.59
lines of backtrace.

The (mostly) raw numbers may help to illustrate this:

- KDevelop:

  crossReferences: 608
     backtrace: 608
       path: 1460
     relatedStatements: 608
       backtrace: 5746
         path: 14246

- KWIVER:

  crossReferences: 7700
    backtrace: 7700
      path: 38668
    relatedStatements: 7700
      backtrace: 222050
        path: 1018377

>From casual inspection of the full JSON, the reason for the difference
can be inferred. KWIVER has many external dependencies that are mostly
consumed by global use of `add_definitions` and `include_directories`.
The large number of these account for the much larger average number of
related statements per target, while project organization accounts for
the larger average line count of backtraces. Combined with a larger
number of total targets, this produces a geometrical increase in the
size of the code model.

The target disparity is itself worth some investigation. Besides
appearing that many targets are duplicated four times, registered tests
also turn into ten targets (for various invocations of sanitizer tools);
thus, a single test target can turn into 40 targets *per test case*
contained in that target. Simply eliminating these reduces the target
list to a much more reasonable 500.

The raw statistics and (xz-compressed) raw KWIVER target list are attached.

(¹ KWIVER also has 42 projects to KDevelop's 1. I wouldn't be surprised
if there is some duplication going on here, especially considering the
disparity in targets-to-artifacts ratios. However, another interesting
note is that a large percentage of the targets - it appears to be well
over half - are CDash related. In particular, each test)

-- 
Matthew

-------------- next part --------------
(root): 1
  inReplyTo: 1
  type: 1
  cookie: 1
  configurations: 1
    (array): 1
      name: 1
      projects: 1
        (array): 1
          buildDirectory: 1
          targets: 1
            (array): 612
              linkLibraries: 239
              name: 612
              artifacts: 268
                (array): 268
              linkerLanguage: 268
              buildDirectory: 612
              fileGroups: 606
                (array): 993
                  language: 537
                  isGenerated: 993
                  sources: 993
                    (array): 2559
                  compileFlags: 537
                  includePath: 537
                    (array): 23633
                      path: 23633
                      isSystem: 13622
                  defines: 537
                    (array): 9368
              linkFlags: 249
              linkLanguageFlags: 167
              sourceDirectory: 612
              crossReferences: 608
                backtrace: 608
                  (array): 1460
                    path: 1460
                    line: 765
                    name: 765
                relatedStatements: 608
                  (array): 5746
                    backtrace: 5746
                      (array): 14246
                        path: 14246
                        line: 7822
                        name: 7822
                    type: 5746
              fullName: 608
              type: 612
          name: 1
          sourceDirectory: 1
-------------- next part --------------
(root): 1
  inReplyTo: 1
  type: 1
  cookie: 1
  configurations: 1
    (array): 1
      name: 1
      projects: 1
        (array): 42
          buildDirectory: 42
          targets: 42
            (array): 7700
              linkLibraries: 390
              name: 7700
              artifacts: 390
                (array): 390
              linkerLanguage: 390
              buildDirectory: 7700
              fileGroups: 462
                (array): 616
                  language: 411
                  isGenerated: 616
                  sources: 616
                    (array): 2252
                  compileFlags: 411
                  includePath: 411
                    (array): 5931
                      path: 5931
                      isSystem: 3418
                  defines: 411
                    (array): 1065
              linkFlags: 390
              linkLanguageFlags: 278
              sourceDirectory: 7700
              crossReferences: 7700
                backtrace: 7700
                  (array): 38668
                    path: 38668
                    line: 30871
                    name: 30871
                relatedStatements: 7700
                  (array): 222050
                    backtrace: 222050
                      (array): 1018377
                        path: 1018377
                        line: 510627
                        name: 510627
                    type: 222050
              fullName: 7700
              type: 7700
              linkPath: 390
          name: 42
          sourceDirectory: 42
-------------- next part --------------
A non-text attachment was scrubbed...
Name: kwiver-targets.xz
Type: application/x-xz
Size: 8056 bytes
Desc: not available
URL: <http://mail.kde.org/pipermail/kdevelop/attachments/20171211/6ae3055f/attachment.xz>


More information about the KDevelop mailing list