Implementing a general language plugin (Reposting from correct address)

[Jonathan Verner]

Hi,

> > However, it seems that the LSP is too limited to support the cool stuff
> > kdevelop does (i.e. semantic coloring).
> 
> This isn't correct as such - all that's needed for semantic colouring is the 
> category and declaration position for each name, and those can be queried 
> through LSP.

Yes, you are correct. I guess I should have written "not suitable for the way
KDevelop does things".

> I don't know if you've seen Emma's blog post, almost certainly the most 
> detailed consideration so far of using LSP in KDevelop:
> https://perplexinglyemma.blogspot.com/2017/06/language-servers-and-ides.html

Yes, I've seen that and I think that the blog was actually one of the reasons I gave up on
the idea of using LSP in KDevelop :-)

> > do you think this approach is workable (so far I didn't run into any
> > obvious roadblocks, but I am new to the codebase)
> 
> I do think it could be made to work, but I'm not sure it's the best approach 
> and there are some issues.

So far, in the limited time I have for this, it seems to work resonably well. I've got 
basic semantic highlighting and code navigation working. Obviously, I haven't yet gone
far enough to hit all the possible issues, but it seems promising.

> This is where, instead of actually answering your questions, I suggest 
> something quite different. :P

:-)

 
> TL;DR - I believe it would be better to implement a simplified "query" API on 
> top of the DUChain, converting existing UI and non-language plugins to use it 
> instead of direct lookups, and then add support for LSP as a backend to that 
> while bypassing the DUChain entirely.

While I agree, I am afraid that this is far beyond me at the moment I have
limited time and limited understanding of the code... One of the reasons I chose my approach
was also that there was a relatively well defined and small area of KDevelop I'd need
to understand.

> The big exception is kdev-clang, where all the declaration and typing 
> information is deduced by Clang and the language plugin is fundamentally a 
> Clang AST -> DUChain convertor.

That is actually the approach I seem to be taking... Basically I define a protocol for sending
a DU-chain like structure to the kdevelop-side of my plugin which then does nothing much 
more then store this in KDevelop. Most of the heavy lifting is done in the C# server, which
uses Microsoft libraries to do the heavy-lifting in constructing an AST and then, using a few simple visitors,
converts it to a DUChain and sends it over the wire to KDevelop. This means that the code
*I* need to write, is relatively simple wrappers and the only problems I am encountering usually
have to do with my poor understanding of either KDevelop or C# :-)

> Unless the proposed protocol is bidirectional (or the DUChain is shared cross-
> process by some other means), plugins for it can't read the DUChain 
> themselves. They must either keep their own copy of the data, reinvent an 
> equivalent framework for analyzing declarations or types, or (most likely) be 
> similar to kdev-clang and rely on an external library.

> That is: plugins for it don't really /benefit/ in their own right from being 
> KDevelop-specific.
> Much as kdev-python etc. struggle with the DUChain in places (see below), all 
> of their internal analysis is based on queries to it; they wouldn't be 
> possible in anything like their current form as standalone utilities.
> In contrast, without read access a plugin has to do all the work on its own 
> and then (lossily) turn it into something the DUChain can understand.

My idea wasn't that plugins would be easier to write because they were written for KDevelop.
Rather I wanted to use the fact that nowadays a lot of languages already provide
much of the heavylifting needed for constructing an AST/DUChain out of the box. 
So I wanted to have an easy way to leverage this --- i.e. to be able to write most of the plugin in
the respective language. Also, it seems more likely, that people wanting support in kdevelop
for language A know language A better than C++ so that it is conceivable that they could
write a plugin in A if they didn't have to touch C++,

> It's important to note at this point that the DUChain is actually quite 
> restrictive, and more so the further you get from C. The implementation is 
> very elegant and performant, but it makes a lot of assumptions about how types 
> and declarations work that can be hard to map other languages onto.
> I spent a lot of time and effort trying to get kdev-python to store more things 
> usefully in the DUChain - types are objects and vice versa, any assignment 
> might be a declaration, and that's very difficult to represent.
> This also seems to be an increasing problem with C++ itself - try anything 
> involving non-trivial templates.

But this restrictiveness isn't as bad once you use the DUChain only as a means
of getting the information to KDevelop, right? (if the plugin stores everything in
its own data structures internally...)

> [sorry, it's midnight and I'm going to sleep. Part 2 hopefully tomorrow].

I am surprised and honored that you had the energy write such long reply to a random person
on the mailing list so late in the evening :-D

> The attached file is a somewhat-relevant IRC discussion from last year. 
> Needless to say I didn't get to working on it. I might be able to this time 
> around...

As I said earlier, I don't feel confident enough to try this, but I agree that in principle
it would probably be the better approach. On the other hand, perhaps I'll eventually
get there..., who knows :-)

Best,

Jonathan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: c#-hilighting.png
Type: image/png
Size: 84596 bytes
Desc: not available
URL: <http://mail.kde.org/pipermail/kdevelop-devel/attachments/20190501/d8ecaf98/attachment-0001.png>