[Kst] memory checks...

[Barth Netterfield]

/proc/meminfo appears to be the only thing under linux.

linux has a command 'free' which uses /proc/meminfo.

We'll have to add some configure tests for /proc/meminfo of the correct 
format.

cbn

On May 27, 2004 03:28 pm, George Staikos wrote:
> On May 27, 2004 14:38, Barth Netterfield wrote:
> > A very real possibility which I have done myself, and made bad things
> > happen.
> >
> > omega% kst -f 0 /data/rawdir/B2K2/ -y bolo1 -y bolo2 -y bolo3 -y bolo4 -y
> > bolo5 -y bolo6 -y bolo7 -m 2
> >
> > I just asked it to read 7x54million sample vectors (+ INDEX).  I meant to
> > add -n 1000000, but forgot.  Each vector is 216 MB, so my 512 MB machine
> > is going to run out of memory and start swapping, then run out of swap
> > and die.
>
> [...]
>
> > The *only* questions are:
> > 	i) under linux, is it possible?
> > 	ii) if so, how.
> >
> > For (i) the answer is clearly "yes".  I can run 'top' and see how much
> > free ram there is, how much ram is being used for cache, and how much
> > swap is free.  Top gets this information, so we can get this information.
> >  Getting this information before the malloc could allow us to make some
> > intelligent decisions about what to do.  (ie, I probably don't want to
> > allocate more than free+cache - if I do my machine will likely be pretty
> > much useless or will just die).
>
>    The problem is that in your case, there is no such thing as "the
> malloc". There are 7 mallocs, each of which is reasonable, but combined,
> become deadly.  So we need to do even more in-depth calculations of how
> much ram is free (which we don't really know since it may have been
> overcommitted to some other app already), and we have to track all of our
> allocations and frees. This will not work for situations of running
> multiple Kst instances too.
>
> > This of course is not bullet proof, (eg, memory gets allocated or freed
> > by some other program/thread in between checking and allocating) but this
> > will be relativly rare, and I bet it would catch the vast majority of
> > problems. Enough problems would be caught to be easily worth the effort.
>
>    What we need then is an algorithm that can guess at how much we can
> safely allocate.