Playing with the QML/JS plugin

[Denis Steckelmacher]

On 04/07/2014 10:38 PM, Sven Brauch wrote:
> Intuitively, I'm skeptical about whether that is going to work. In other
> languages (when I say that, I mostly speak about python ;) the Expression
> visitor is used to get the type of an expression in a variety of places. For
> example, when I have a = 3 I create an expression visitor to visit the 3, then
> assign the type to the variable. However, when I have foo.append(3), then
> maybe I want to add the type of 3 to the list content type of foo. Having a
> tool which allows to do "determine the type of this expression and return it"
> is very flexible and usable in a variety of situations.
> You might be able to replace that flexibility partially by what you're doing
> now -- storing the types of nodes in those nodes themselves. I can't really
> judge whether this is "good enough", though. For example, for code completion
> it sounds a bit difficult to re-use (you don't want to run the declaration
> builder on the expressions you handle in completion).
>
> What motivated you to remove the expression visitor? Why is it an advantage?

The main reason is that I wanted to clearly see what does what and how 
everything works together. For instance, function declarations were 
handled in DeclarationBuilder (which declared the functions), 
ExpressionVisitor (which created FunctionTypes) and finally 
ContextBuilder (which opened/closed the inner context of functions). I 
started with no technical argument for the replacement of 
ExpressionVisitor, I just wanted to play with the code, and now that I 
understand how everything works, I would have no problem reverting to an 
ExpressionVisitor-based solution.

Using an ExpressionVisitor is cleaner, and may even be required if the 
completion code needs to access types and has no access to a parse 
session (I store the types of nodes in a QHash<Node*, AbstractType::Ptr> 
in ParseSession), but my experiments made me wondering how 
ExpressionVisitor behaves with respect to these points:

* The type inference system has to build types (functions, objects), and 
some of these types require an internal context. Creating these types 
and contexts may be easier in a subclass of AbstractTypeBuilder and 
AbstractContextBuilder, even if it is possible to split the creation of 
the types and the creation of the contexts, like what was done for 
functions (their context is created in ContextBuilder, their type in 
ExpressionVisitor).
* ExpressionVisitor only computes the types of what it is asked to, 
which is good, while my version of DeclarationBuilder computes the type 
of every node. One advantage of DeclarationBuilder, though, is that it 
has a strict O(n) complexity : every node is visited at most one time, 
and types are propagated from bottom to top. ExpressionVisitor may be 
called several times on nested nodes and will therefore re-explore some 
of them several times, thus re-creating types.
* There is a clear separation between the creation of the types (when a 
new type is encountered, it is passed to ParseSession::setTypeOfNode) 
and the use of types (ParseSession::typeOfNode does never create a 
type). This may be important in order to avoid the code-completion code 
to create types where it had to reuse existing ones, but I've still to 
look closer at how you implemented that in the Python language plugin.

>> As the type inference system and the declaration builder are
>> now intertwined, I also had to declare variables and QML fields in
>> endVisit() instead of visit(), because the types get known only when
>> endVisit() is called. I would appreciate any comment on what I've done :-) .
 >
> To me it seems like all those problems are generally avoided by having the
> expression visitor separate. Because then when you visit e.g. an assignment (a
> = 3) you can just do
>    ExpressionVisitor v(assignment->rhs);
>    lhs_decl->setType(v.type());
> or so.

Yes, the biggest advantage of ExpressionVisitor is that it allows some 
sort of "look-ahead". You can compute the type of a variable before 
entering its node. My solution requires that the expression has been 
visited in order to have its type computed and stored, so you have to 
declare variables in endVisit() in order to be able to use 
m_session->typeOfNode(node->expression);

>
>> * Can I change the type of a declaration once it has been closed (for
>> instance, by using getDeclaration(...)->setType(new_type))? This would
>> allow me to guess that a in "var a; a = 5" is of type int, and it would
>> be especially useful for setting the type of function parameters once
>> they are used in a context were their type becomes evident.
> Yes, you can do that at any time. Make sure to look at how unsure types work,
> too, those are important for dynamic languages ;)

Wonderful! This may even allow me to declare variables with an unsure or 
mixed type in visit(), and then to set their correct type in endVisit().

>
>> * Is it possible for a declaration to have a source range outside the
>> range of its parent context? It would be very useful to implement things
>> like "var a = {}; a.b = 5;", with b being a field declared in the {}
>> context created in the first statement.
> I'm not 100% sure but I think no, not so easily. I think we already discussed
> this as a possible issue a while back in IRC, no? Much of the duchain, esp.
> its memory management (cleanup logic) is built around having everything nicely
> ordered and nested range-wise. Python has an ugly hack for allowing pretty
> much exactly this (declaring class members from outside the class context). We
> should investigate if we can find a more proper solution some day, now that
> more languages than just one need this.

Yes, we talked about that on IRC. I may have a solution for this 
problem: context can have "imported parent contexts", that may be used 
in situations like that:

var v = {a: 1, b:2}

v.c = 3;
v.d = 4;

Here, the declaration builder creates four contexts: the one enclosing 
{a: 1, b: 2}, another one enclosing c ("v.{c} = 3") and in which c is 
declared with type int, another one for d, and finally an empty context 
(which may span the "=" sign if zero-length contexts are forbidden).

The type of v is set to be a StructureType, whose inner context is the 
empty context. This empty context has the three other ones as imported 
parents. In this empty context, a, b, c and d thus become visible. I 
think this would allow "v." to offer a, b, c and d as completions. If a 
third assignment is encountered, then it should suffice to create a new 
context, declare "e" in it and then import this new context into the 
empty context of v. If c is assigned twice, this can even be detected by 
using empty_context->findDeclarations("c").isEmpty(), and c will not be 
re-declared.

The only problem that may arise is that I must create the empty context 
at the declaration of v, but I need to add imported contexts to it 
afterward. Is it possible to call DUContext::addImportedParentContext on 
a context that already has a parent? (I've read that I need to check 
TopDUContext::isInChain or something like that)

Denis Steckelmacher