Python parser ranges

[Sven Brauch]

Hi,

there's a small python script which reads python code from stdin and
writes an AST in XML format to stdout, using python's "ast" module for
parsing. This XML is then read into a C++ AST structure by the plugin.
Not very elegant, but it works and it isn't too slow either (i think
the parser we had before was slower). An example for such an XML ast
can be found here:
https://projects.kde.org/projects/extragear/kdevelop/experimental/kdev-python/repository/revisions/master/changes/example_ast.xml
This currently relies on python being installed (which is likely to be
the case if you're attempting to use a python IDE). In case I wanted
to fork the python parser, I would need to ship with pretty much of
the official python source code... quite screwy. In this case, it
would at least be good to skip the XML step and connect the parser
code directly to the plugin code... but unfortunately it's written in
C and I don't really have a good idea how to connect those two nicely
without doing a lot of work (and thus binding the plugin to one
specific version of the parser)...

To repeat that, there's a small change which would be required to the
python parser code (which as far as I can see would only have positive
side effects, if any) if I want to avoid huge amounts of (potentially
slow, of course) regular expression hacks: it would need to tell me
that for the line "foo.bar", "bar" starts at column 4 -- instead of
column 0, which is what it currently claims.

Bye,
Sven

2010/12/25 Niko Sams <niko.sams at gmail.com>:
> On Sat, Dec 25, 2010 at 16:37, Sven Brauch <svenbrauch at googlemail.com> wrote:
>> Hi,
>>
>> the python parser is cool, but it doesn't give me quite all the range
>> information I need. Sometimes, it just gives pointless crap (like, for
>> foo.bar.baz it tells me that the column offset for baz is 0).
>> This is easy to fix in the python parser (it's like removing 2 lines),
>> but it doesn't look like they would change that (see
>> http://bugs.python.org/issue10769).
>> If I want to do this in the plugin, it's terribly complicated, as I
>> kind of have to write my own lexer and stuff... Alternatively, I could
>> fork the python parser, as it's really only two lines which need to be
>> changed...
>>
>> What would you recommend?
>>
> I didn't follow the recent developments in python support, could you
> write just a few lines
> how this all works?
>
> In theory I would prefer a fork that changes only very little code and
> so can easily get updated
> over some custom code that has to be maintained completely by ourselves.
>
> Niko
>
> --
> KDevelop-devel mailing list
> KDevelop-devel at kdevelop.org
> https://barney.cs.uni-potsdam.de/mailman/listinfo/kdevelop-devel
>