kdev-pg: Port to Flex / %member declarations

[Jakob Petsovits]

Hi Roberto and list,

here's something I've been trying out, and I'm wondering if you like it.


- One itch:

kdev-pg writes all of the variables in the parse tree by itself, those are 
merely references to some tokens. That's ok for most AST nodes, but 
insufficient for others.

For example, in my Java grammar I want to have a counter for the "multiple 
brackets" rule ("[][][]" - count: 3) instead of just having a list of 3 left 
brackets in the node. Other example: I'd rather store the modifiers 
("public", "static", "final", etc.) inside one modifier flag variable instead 
of storing one token for each of them. There are also loads of possible enum 
variables that would fit just fine into the AST.

All of this makes for easier access to the AST after parsing.


- Another itch:

Currently, I can't make configuration variables a member of the parser class. 
For the Java grammar, that would be the Java compatibility mode (1.[3,4,5]) 
which is currently stored in a static class variable of a settings class, and 
that's hardly better than a global variable.

I'd strongly prefer the compatibility mode belonging to the parser class 
itself, together with its accessor methods.


- Approach:

The main patch in here (kdev-pg-member.diff) adds a %member declaration to the 
syntax of a grammar file, where every %member occurrence can hold one of:
- any public, protected or private declaration code,
- code inserted into the constructor or the deconstructor.

That code is then inserted into the body of either the parser class or an AST 
node struct, scratching both itches at once.

This patch depends on the kdev-pg-flex-port.diff patch which replaces the 
hand-written lexer with a Flex one. This makes recognizing keywords and using 
lexer states much easier. The Flex port patch doesn't change any of the 
functionality though, error messages are also still working.

With those two patches applied, I can make use of %member declarations in the 
Java grammar (kdev-pg-member-java.diff) in order to replace many 
token-storing variables with enums, bools or ints.
Makes the AST much prettier. Also makes for happy grammar authors.


- Syntax by example:

%member ([target class/struct]: [where to go]) [: code; :]

%member (parserclass: public declaration)
[:
  enum java_compatibility_mode {
    java13_compatibility = 130,
    java14_compatibility = 140,
    java15_compatibility = 150,
  };
  java::java_compatibility_mode compatibility_mode();
:]

%member (parserclass: private declaration)
  [: java::java_compatibility_mode _M_compatibility_mode; :]
%member (parserclass: constructor)
  [: _M_compatibility_mode = java15_compatibility; :]

%member (optional_declarator_brackets: public declaration)
  [: int bracket_count; :]

%member (equality_expression_rest: public declaration)
[:
  enum equality_operator_enum {
    op_equal,
    op_not_equal
  };
  equality_operator_enum equality_operator;
:]


As this patch is a bit more intrusive than my previous ones, I'd like to have 
it reviewed before committing it. What do you think?

Bye,
  Jakob


Patches:
http://stud4.tuwien.ac.at/~e0127236/devel/kdev-pg-flex-port.diff
http://stud4.tuwien.ac.at/~e0127236/devel/kdev-pg-member.diff
http://stud4.tuwien.ac.at/~e0127236/devel/kdev-pg-member-java.diff

P.S.:
Amilcar, why is my mail with real attachments suppressed without any sign of 
waiting for moderation or deletion? That's kind of awkward.