On Memory consumption of KDevelop-PG

[Andreas Pakulat]

On 19.12.09 18:44:22, Milian Wolff wrote:
> On Saturday 19 December 2009 10:49:47 Andreas Pakulat wrote:
> > > - is the ducontext pointer on _every_ node really neccessary? Imo it only
> > > makes sense for functions, classes and top-statements... At least the
> > > parts in PHP where we use them can be changed to pass a currentContext(),
> > > instead of using the ast-member.
> > 
> > No most probably this is not needed. However its a bit more cumbersome
> > to get members into specific AST nodes only IIRC.
> 
> No, not at all, see:
> http://techbase.kde.org/Development/KDevelop-PG-
> Qt_Introduction#defining_additional_variables_for_the_parse_tree

That is more cumbersome than having a single member-declaration that
applies to all AST nodes. I wasn't saying its too cumbersome to be
useful.
 
> > > - finally, maybe the hardest part: Can't we compress the AST _while
> > > building it_, i.e. can we somehow drop all these useless
> > > logicalAExpression -> logicalBExpression -> logicalCExpression -> ... ?
> > > Imo we should at least make that one node and work with members (i.e.:
> > > logicalExpression->type == LOGICAL_XOR) and than a ptr to either a "real"
> > > (not-logical) expression or to a nested logical expression... Maybe that
> > > could save us some more MB...
> > 
> > As I said above, taking some extra memory during the parsing process is
> > ok if you can properly free it again. So what might be a valuable
> > approach is to either drop the AST stuff completely (AFAIK C++ does
> > that) or generate a compact AST. I'm doing that for QMake support
> > (though more for the "easy to work with" reason that memory consumption)
> > and it was mostly "dumb typing" work after having defined an compact
> > AST.
> 
> I remember that, you once told me that already. I'll have to look at the QMake 
> sources. But to clarify: You are still using a parser generator? As I said 
> above, I don't want to remove that.

Yes, there's a kdev-pg-qt generated parser which creates an AST. Then in
a second step this AST is walked over and a new AST created out of it,
with quite some indirections removed. So it'll still consume much memory
first during parsing, but then the memory consumption will drop (how
much I don't know). The same is done for the python language support,
but there indeed each AST node has a context. And even "worse" each node
already has line/col info about where it starts and ends (as qint64).

Andreas

-- 
Beware of low-flying butterflies.