Memory consumption with large projects

[David Nolden]

Am Donnerstag 26 November 2009 00:09:55 schrieb Milian Wolff:
> On Wednesday 25 November 2009 23:50:04 Alexander Dymo wrote:
> > I tried to open a project with roughly 500 000 lines of C code (about
> > 180M on disk) and got some strange issues with memory consumption.
> >
> > During parsing (no persistant storage used):
> > - 3.8G used right after parser is started and before it showed 1% in the
> > progress bar
> > - it then looked like that memory was free'd and KDevelop started
> > actually to parse
> > - 2.2G occupied after parsing
> >
> > During loading from persistant store:
> > - just 200M was used (at any time)
> >
> >
> > Do those numbers make sense? I'd be most curious to find out why KDev
> > takes so much memory.
> 
> Hey Adymo! Off to fixing bugs in Ruby again? :)
> 
> This behavior is known. Afaik it is suspected to be due to (at least in
>  some degree) the kernel/whoever not coping well with lots of small frees
>  => memory fragmentation.
Me personally I have never suspected this. ;-)

> We should probably start to use memory pools here and there, at least that
>  was the "solution" I have in my memory.

We should find out what exactly is eating up the memory that much, massif 
should be able to tell us. I've already fixed quite a few memory hogs, but 
none of them has been 'the one' until now.

Greetings, david