On Memory consumption of KDevelop-PG

[Milian Wolff]

Hey all!

I did a massif run with duchainify on Mediawiki, and well as far as I can see 
there is no apparent memory leak.

http://mwolff.pastebin.com/f3581a8d6

What bugs me is the peak... See also:
http://mwolff.pastebin.com/f4a66d631

I mean, it's pretty clear why we have a pretty high memory consumption:

- every visited node gets an Ast, even those that are just general helpers 
(AST == Tree.. duh)

- every Ast has two qint64 + a pointer _at least_, meaning: sizeof(AstNode) == 
32, most "specialized" nodes have at least one pointer to a child element, so 
lets say sizeOf(node) = 36

- the phpfunctions file alone has 1.5 mio nodes (and this file is pretty simple 
imo, no logical expressions or stuff like that which will likely blow up the 
node number greatly, I mean the massif peak is somewhere later, dunno how to 
find the exact file... but it's _not_ the internal file)

so these nodes alone take up (36*1.5E6/(1024^2)) = 51.5 MB for the 
phpfunctions file...

Can't we reduce that somehow?

- is the ducontext pointer on _every_ node really neccessary? Imo it only 
makes sense for functions, classes and top-statements... At least the parts in 
PHP where we use them can be changed to pass a currentContext(), instead of 
using the ast-member.

But what about the context-builder etc. I mean esp. setContextOnNode and 
contextForNode... Afaik they are only relevant to nodes that actually 
correspond to contexts, right? I.e. to functions & classes.

Saving these 4 Bytes per node would decrease the size by 10%...

- do we really want to support gigantic source files, or why do we make 
startToken & endToken a qint64? Just making them int would save us 8 Bytes, 
i.e. 20%

- finally, maybe the hardest part: Can't we compress the AST _while building 
it_, i.e. can we somehow drop all these useless logicalAExpression -> 
logicalBExpression -> logicalCExpression -> ... ? Imo we should at least make 
that one node and work with members (i.e.: logicalExpression->type == 
LOGICAL_XOR) and than a ptr to either a "real" (not-logical) expression or to 
a nested logical expression... Maybe that could save us some more MB...


-- 
Milian Wolff
mail at milianw.de
http://milianw.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kdevelop-devel/attachments/20091219/ba60eb4c/attachment.sig>