[Kst] memory checks...

[George Staikos]

On May 26, 2004 16:03, Andrew Walker wrote:
> To catch every case of a failed memory allocation would involve
> using try catch around every function that performs a memory allocation,
> including (presumably) all Qt and KDE code. In my opinion this would
> be the ideal situation - programs should not crash under low memory
> conditions. I recognize this is a lot of work but it is something that
> could be done incrementally.

   The problem is that the code internal to Qt and KDE don't do this anyway.  
Actually I don't think too many toolkits do handle this completely.  If an 
exception is thrown in the middle of a call too, for instance, KMDI, I don't 
think it will be able to recover even if we do.  The best we could hope for 
is to do like KMail, write our files to a hardcoded location on disk and 
exit.   That's not much of an issue in Kst since Kst is generally used for 
reading from disk, not writing to it.  I don't think "recovery" is really a 
possibility, only notification of major problems.  This can be dealt with 
using a signal handler I think, though it could possibly crash too.

> Failing that, we should at least do a confirmation of the memory
> allocation for major allocations; such as new vectors.

   How do you intend to do this?  If you want to implement it and make it 
compile-time enabled, that's fine with me, but I don't know of a reliable way 
to do this without going into kernel space on Linux.

> Requesting a large memory allocation will fail with a std::badalloc
> exception thrown. You can test this in code. Just request a buffer
> larger than your memory space (including swap).

   I don't think requesting buffers greater than the total memory plus swap is 
a realistic event.  More likely there will be 5-10 smaller buffers that may 
add up to more than the amount of ram+swap.  It would probably be more 
effective to just determine the amount of memory+swap in the system, remove 
200MB or so, and keep track of the vectors we allocate.  Then we can just 
tell the user if he tries to load too much data.  I think that would be far 
easier to implement.

> If portability prevents us from using particular features then we
> just ifdef them out in those cases.

   This makes things very messy for very little gain I think.

> People most certainly will use Kst under low memory conditions.
> Perhaps not intentionally. Clearly they will not expect Kst to crash,
> and will be less than happy if it does.

  I don't understand why.  It's used for scientific work.  It is run on 
machines designed to do the work that they are doing.  I don't see why they 
would be using machines that are incapable of doing the work necessary except 
in rare occasions.  Anyway, it's still possible that they could be, but that 
doesn't circumvent the overcommit problem.

   So, the reasons why it won't really work or is too much trouble:
1) Linux, primary target platform, overcommits
2) Not all techniques for this are supported by the various compilers, or they 
need special switches -> will require hacks to the build
3) We will almost certainly get a crash inside Qt or KDE as a result of it
4) The userbase generally has equipment designed to handle the amount of data 
they are processing in Kst
5) Only protecting the big mallocs is a bit insufficient since we do many more 
mallocs by number inside the libraries.  We could get close to the crash 
limit but not quite, then try to open a dialog, and *poof*, there goes Kst
6) It's a big maintenance problem

-- 
George Staikos
KDE Developer			http://www.kde.org/
Staikos Computing Services Inc.	http://www.staikos.net/