GSoC 2010: Making KDevelop suitable for large C projects

[Nitin Gupta]

On 03/30/2010 12:37 AM, Andreas Pakulat wrote:
> On 29.03.10 23:16:06, Nitin Gupta wrote:
>> On 03/29/2010 06:03 PM, Milian Wolff wrote:
>>> Nitin Gupta, 27.03.2010:
>>>
>>>> For past few months, I have been working on a simple IDE (called KXref)
>>>> based on Sublime UI framework, to allow working with large C based
>>>> projects (Linux kernel in particular): http://code.google.com/p/kxref/
>>>>
>>>> However, I soon realized that it will be much more useful and productive if
>>>> all this can be developed as plugins for upcoming KDevelop4. My initial
>>>> attempt to get it working for Linux kernel failed badly primarily due to
>>>> lack of cscope, or better GNU global, integration (built-in background
>>>> parser is too slow for such large projects). There are many other factors
>>>> which make it unsuitable for kernel projects (as detailed below).
>>>>
>>>> Unlike popular belief, an IDE for kernel projects (IMHO) is highly
>>>> desirable. Trying to understand such huge and complex codebases is almost
>>>> impossible otherwise. Plain vim+ctags is often not sufficient.
>>>>
>>>> I'm planning to submit this as a GSoC proposal. I would be thankful for any
>>>> feedback.
>>>
>>> My personal opinion is: One can integrate each and every tool there is for 
>>> KDevelop, but I'd much prefer improving the performance of the build in parser 
>>> instead of using yet another tool for the job... 
>>>
>>> I found quite a few places to optimize over the weeks, and I bet there are 
>>> more.
>>>
>>
>> Built-in parser seems to be slow beyond repair. Following is test summary:
> 
> I suggest you back that statement up by some knowledge about the
> codebase. Making such blanket statements won't get you much friends
> around here ;)
>

Yes, I should have gathered proper profiling data but the numbers are still very
disappointing. Argument that its all done in background with preference
given to opened files isn't too nice -- with large C project containing 20k - 30k
files, the parser (in its current form) can take hours pumping CPU at nearly 100%.
Maybe improving in-built parser speed is the way to go but it was too tempting
for me to go for already optimized solutions like gtags even if they could not
provide as much of information as in-built parser.

Maybe we should have options for in-built parser like: more comprehensive parsing vs speed.
I think this will quickly make KDevelop suitable for large projects while it is gradually
optimized.


 
>> I have not yet started digging into code but I guess above stark difference in gtags vs in-built
>> parser speed is due to much more contextual information gathered by the latter (like usage of a
>> local variables withing functions etc), but now quite sure.
> 
> I don't know gtags, but the amount of information gathered by ctags is
> mostly useless to me. The result is only suitable for doing some basic
> completion, not for code-navigation or finding out how code works.
> 
>> At the same time, use in-built parser to provide rich local context
>> information -- like highlighting usage of a local variable within its scope -- for currently opened
>> files only.
> 
> You can have that already today, simply disable project parsing.
> 

oh, good to know :)

>> Apart from faster parsing, essential components for supporting large C projects are:
>>  - Graph Viewer: Use information from gtags query to quickly generate call graphs
>>     - Check if existing graph viewer can be extended to use gtags or a new one needs to be developed.
> 
> This is available as playground plugin already based on duchain,
> automatically working for all language plugins we have.
> 
>>  - Query Viewer: Again use gtags for queries and show results as generated by gtags
>>     - Current 'Code Browser' window does not show query history. A tabbed query result window is needed
>>       to maintain query history (very important).
> 
> What exactly do you mean with "query"?
> 

I meant queries like "find callers of this function", "find definition of such function" etc.

Thanks,
Nitin