Tiles, Mmap and Crashing

[Bart Coppens]

So because of the endless streams of complaints that Krita tends to crash 
after it uses about 1.8G of swap, I started to rewrite the system a bit. Now 
instead of just letting a memory mapped region sit in the memory (with the 
kernel instructed that we won't need it if it's 'swapped', etc), I tried to 
make it such that it actually munmaps tile regions that are unswapped. As a 
positive point, this makes the virtual memory usage somewhat lower, and 
allows us (hopefully?) to exceed some 4G virtual memory border for swap files 
on 32bit systems, I guess.

This brings some rather unfortunate consequences with it as well, 
unfortunately. Since I now actually munmap stuff when swapping out, pointers 
to swapped tiles are no longer _valid_ (rather than just not being expected 
to being used by the kernel, and still valid to read/write to). In my new 
code I'm trying to set the pointers to 0 if it's swapped out, so that code 
will crash, but at least it'll be clear why. But those pointers can be cached 
basically anywhere during the swapping, resulting in segmentation violations 
at places where it's not expected (possibly I already found such an 
occurrence unfortunately).

For normal use this is no problem, since most code uses iterators to access 
tiles, and using iterators _ensures_ the user that the tiles in the iterator 
range won't be swapped away. The problem lies in the pixelPtr(), pixel() 
functions (and possibly other methods I didn't find yet). They just return a 
raw pointer, which we cannot check if it's in use or not.

Would anyone object to us having a pixelPtrSafe, returning some kind of Shared 
Pointer to the data, that ensures data availability as well? Since 1.6 isn't 
allowed to remove methods, the old functions can't disappear. (Or if you have 
a better idea, please shout!).

Some quick experimental results: with my first caching settings, I was able to 
get Krita to have about 2.7G of tiles being used (and then my disk ran out of 
space, which oddly wasn't catched by my safety code), and still it didn't 
crash due to out of memory. The bad thing is that I then changed the 
maxTilesInMem+swappiness, and now it mysteriously crashes in setDefaultPixel. 
So it's not yet ready for committing anyway, but this way you know it's 
somewhat possible already ;-)

Bart Coppens