Clang tutorial

[Milian Wolff]

Hey all,

I'm forwarding this to kdevelop-devel at kde.org - please direct the future 
replies there so others can read it too and chime in if desired.

On Saturday 08 February 2014 16:47:14 Michael Ferris wrote:
> Hi, my name is Michael, we haven't met yet, but I am one the guys that will
> be working on Clang for kDevelop for the next weeks.
> 
> We have started working on it a bit. So far we enabled renaming of
> variables, however, since this is not language dependant, we only needed to
> hook the refactor menu to the basic refactor component which uses the
> DUChain to do the renaming.

Cool, is the patch already up for review? I might have missed it so far. 
Otherwise, push it to reviewboard and I'll have a look.

> Now, that said, we would like to know how to start using Clang.
> 
> If we start from clanglanguagesupport.cpp, what are the links with clang
> that we can use to read/modify the code ?

Using the existing language-agnostic framework is perfectly fine for some 
features like the above. For other parts leveraging clang instead might be 
better, but thats left to be seen and decided on a case-by-case study.

> Also, if you could give us a quick overview of the main components of clang
> and their relations to one another and the current code, that would be most
> appreciated.

So, from clang itself we only use the clang-c API, see esp. 
/usr/include/clang-c/Index.h 

This is used mostly in two places so far:

#1 - the DUChain integration.

The DUChain in KDevplatform is a strange name for what basically is a cache of 
data from language plugins. It holds locations and types of declarations, 
contexts, uses, ... 

In the kdev-clang/duchain folder, you'll see our current approach in 
translating the data we get from clang-c into the existing data structures of 
the DUChain. The ClangParseJob is run for every file you edit or which is part 
of a project loaded in KDevelop. There a ParseSession is created which 
builds/updates a clang translation unit. This is then run through the 
TUDUChain builder which extracts the contexts, declarations, types and uses 
out of the TU and stores them in the serializable data structures of the 
DUChain.

Once we have that done, stuff like context browsing and code highlighting are 
extracted from these data automatically by code in KDevPlatform.

If you spot any bugs there, such as wrong types in a context browser popup, or 
bad code highlighting, you'll want to touch this part. Otherwise, this is 
mostly done I think and not so much is left to do. Some things I can think of 
are:

## Forward Declarations

Afaik, we don't special-case forward declarations at all so far. This could be 
useful, to get the nicer navigation popups which offer you to jump to the 
"actual" declaration etc.

## Qt Integration

This is probably pretty hard, but our existing C++ plugin has some special 
code for improved Qt support. Qt uses the moc, a pre-preprocessor for some 
special code generation and the actual compiler won't see most of the 
"special" macros. Olivier Goffart did some investigations in that regard 
though, and showed that clang can be made to work with this, see 
http://woboq.com/blog/moc-with-clang.html . It's left to be seen how we can 
achieve the same and whether that data is then also accessible for us via the 
clang-c API. Without that, support for signals/slots and other core Qt 
features will suffer.

#2 - Code Completion

This is in the kdev-clang/codecompletion folder, mostly see context.cpp 
therein. Given a cached ParseSession from a previous ClangParseJob run, we 
invoke the respective clang-c API to get code completion results. The output 
is then again translated into KDevelop-specific data structures. This is so 
far quite minimalistic but works reasonably well for the simple code 
completion.

Here, quite a few things can and should be done by you guys I hope. Mostly, 
this is implementing features we have in our current C++ KDevelop plugin, but 
which are not provided by the clang API as of yet. Here's a list of things 
that are missing:

## Implement Function

If a function is only declared but not defined, we offer a "implement 
function" helper item in the code completion. Try it out in our old C++ 
plugin:

class foo { void bar(int i); };
// invoke code completion here

You'll see a "implement foo::bar" code completion item which when executed 
inserts the boilerplate code:

void foo::bar(int i)
{

}

This should be done in kdev-clang as well. So, investigate how to get this 
information from clang-c or based on our DUChain data, and copy/reimplement 
the feature from kdevelop/languages/cpp/. See esp. the 
ImplementationHelperItem.

## Override Function

Invoking code completion in a class body which inherits from a polymorphic 
class, add items which offer you to override a virtual function of a base 
class. Here, its left to be seen how to implement it, i.e. based on the 
DUChain or whether clang offers this information to us. This feature is 
implemented also in the ImplementationHelperItem in our existing cpp plugin.

## Missing Includes

When you try to access a class or function in a file but have not yet added 
the respective include statement for the header which declares it, KDevelop 
should offer you to insert the missing include. See the files with that name 
in kdevelop/languages/cpp. This probably is mostly copy'n'pasting of the code 
(and cleaning it up), as finding the files cannot be done by clang, but only 
via our DUChain cache.

## Call Tips

When showing code completion inside a function call or constructor, we want to 
show information about the functions that can be called. I.e. what functions 
are viable for the previously written arguments, what overloads exist and what 
their signatures are.

This is probably all thats missing from code completion so far. Did I forget 
anything?

#3 - Assistants / Refactoring

Another neat feature in KDevelop which we want to have also in kdev-clang, are 
the assistants. See also kdevelop/languages/cpp/codegen/

## Rename

When you change the name of a local variable declaration, which was used 
somewhere, we offer an assistant which can be invoked by "Alt + 1" - this then 
triggers a rename of all uses of the old variable.

## Adapt Signature

Similar to the above, if you change the signature of a function 
declaration/definition, we offer you to adapt the corresponding 
definition/declaration signature.

## Declare function/member/variable

When you use some undeclared function/member/variable, we offer you to create 
a private/public/local declaration of that as appropriate.

## Rename File

When you rename a class and the file it was declared in matches the old 
identifier, we offer you to rename the file in accordance.

## $neat feature

Afaik, clang comes with a lot of interesting refactoring tools. It would be 
cool to integrate them somehow.

#4 - Other Stuff

Another feature which is currently missing, is special support for macros. See 
our old's cpplanguagesupport.cpp and the specialLanguageObject* code in there. 
It adds code browsing support when the cursor is inside a macro expansion and 
shows the macro, its definition and the actual expanded preprocessed text. 
Would be cool if we can keep that somehow when we get that info from clang. 
See also cppduchain/navigation/ - esp. the includenavigation* and 
macronavigation* files.

#5 - Clang Format

Unrelated to the actual language support plugin, is adding a code formatter 
which uses clang-format.

#6 - tl;dr;

So, this was a long mail but I hope this gives you some ideas on what you can 
work on. Generally, pick what fits you best and whats interesting to you. Try 
to use our old C++ plugin for some of the features above to see how they 
should work. If you have other nice ideas for what to do, go for it! If you 
have any problems, ask on our mailing list!

Bye and have fun!

-- 
Milian Wolff
mail at milianw.de
http://milianw.de