[Konsole-devel] [Bug 176974] High memory consumption when using Konsole for a longer time.

[Neil Skrypuch]

https://bugs.kde.org/show_bug.cgi?id=176974

--- Comment #70 from Neil Skrypuch <ns03ja at brocku.ca> ---
Created attachment 73539
  --> https://bugs.kde.org/attachment.cgi?id=73539&action=edit
malloc() testing program

Just for posterity, the observed "memory leak" in comment 24 is not a memory
leak and deserves a proper explanation.

Try the attached program:

g++ testmem.cpp
./a.out

You will want to run something like watch -n0.3 "ps aux | grep a.out" in
another konsole while it's running.

For the non-developers, I'll explain what exactly happens here.

The program allocates half a gig of memory in a single chunk and writes to the
memory (step 1), then frees it (step 2). This memory will be immediately
returned to the system.

Next, it will allocate half a gig in 32k chunks and write to each chunk (step
3), then free each of the chunks (step 4). This memory is not immediately
returned to the system.

Steps 5 and 6 are a repeat of steps 3 and 4, but they forcefully return the
used memory to the system, and so you see the memory usage drop again.

So what happened, why wasn't the memory returned to the system after step 4, it
must be a memory leak, right? Wrong. Run the program in valgrind and you'll
find that there's no memory leak, and a cursory viewing of the source will
support that observation.

First of all, it's important to note that memory allocations are (very)
expensive operations, and a typical program does A LOT of them, and more
specifically, the typical program does a lot of small memory allocations which
last for only a small fraction of the lifetime of the program.

Consequently, it's a worthwhile practice to try and optimize for this very
common use case of many short lived mallocs, and glibc's malloc implementation
does exactly that. For memory allocations below a configurable size threshold,
they are drawn from a pool of memory entries which are allocated as needed and
cached for later usage, even after being free()d. This means that when we
repeated the existing steps 3 and 4 in as steps 5 and 6, the 2nd round of
malloc()s would be essentially free (and as you can see, there is no increase
in memory usage), instead of being very expensive like the first time.

Memory allocations of a size larger than the configurable threshold are not
drawn from a special pool and are not cached for later use, because such larger
memory allocations are rarely made in any meaningful volume within the same
program execution context and have a substantial memory cost associated (if
they were to be kept around like the smaller chunks).

The malloc implementation in glibc is (very much by necessity) a very general
malloc that performs well in a wide variety of uses both in terms of CPU time
spent and memory usage. It is also a very tunable malloc implementation.
Sometimes there are degenerate cases that a general implementation cannot
handle well, and that's why functions like malloc_trim() exist.

In this case, it might be appropriate to call malloc_trim() after closing a
tab/konsole with a large (or non-unlimited) scrollback buffer, as konsole can
have relatively unusual bursty patterns of memory usage, which if allocated in
small chunks, will not be returned to the system for awhile.

-- 
You are receiving this mail because:
You are the assignee for the bug.