Third iteration of QMake parser, looking for a parser generator

[Roberto Raggi]

Hi,

Il giorno 05/lug/07, alle ore 02:44, Andreas Pakulat ha scritto:

> On 03.07.07 03:35:59, Andreas Pakulat wrote:
>> I'm going to give coco/R a more thorough test in the next days,  
>> but in
>> the meantime I welcome any comments.
>
> So I played around with coco/R now and a big plus is that I was  
> able to
> use QString/QChar for the lexer only by changing the template's.  
> I'm not

cool!

> switch to kdev-pg + handwritten lexer. Along the way adding a
> token stream that uses wchar_t* to properly support unicode.

Yeah, that hardcoded "char*" in the token stream is just wrong :-) an  
alternative solution is to kill the "text" field from the token_type  
declaration and provide a tokenText that works with QString/ 
QStringRef. For example,

   struct token_type
   {
     int kind;
     std::size_t begin;
     std::size_t end;
     // ### KILL THIS DECLARATION     char const *text;
   };

in your subclass

   QStringRef tokenText(size_t index) const {
     kdev_pg_token_stream::token_type t = _M_token_stream->token(index);
     return QStringRef(&contents, t.begin(), t.end() - t.begin 
());    // contents is a QString
   }

See? unicode support and fast token manipulation because you don't  
have to create(or copy) QString(s). Let me know if you decide to  
write a hand-written lexer. I can send you an example of how I like  
to write scanners, you can look at it and decide if you want to use a  
similar approach.

ciao robe