Thinking about large files.

[Bart Coppens]

On Monday 13 June 2005 21:23, Boudewijn Rempt wrote:
> One think Krita can not handle are really big files. A 125MB tif file is
> loading into memory in one go, making Krita take about 289MB of memory --
> with no image loaded, Krita takes a more reasonable 41MB. And during
> loading, Krita needs 800MB, and that's a bit much.
That seems actually reasonably acceptable. I wouldn't have thought krita would 
consume much more (we can't really blame krita for the memory usage during 
loading).

> (although we should store, when an image gets above a certain size, an .png 
> file of the rendered image that we can quickly load and display, and then we 
If I recall correctly, this is what the gimp does. (I thought it actually cut 
the entire render in pieces again, to let them load and save the 
(re)renderings faster, but I'm not sure about that).

> This must be done somewhere in the datamanager code. If an image reader
> (whether it's a filter or Krita's own file format reader) indicates that it
> supports random access, then a file handle and a the reader object is
> passed to the tile manager.
> The tile manager either does nothing until it gets a request for a certain
> chunk of data (through one of the read functions or the creation of an
> read/write iterator), and only then it starts reading and decoding image
> data. Or, the tile manager starts a background thread and begins converting
> the alien image data to Krita tiles, carefully caching them in a temp file.
I think we should actively let krita load the file in the background. That 
way, the file is loaded much faster, thus giving the user faster a completely 
loaded image to work with.
Also, it seems that when loading images with ImageMagick, IM first loads the 
entire image into its own memory (I think, but it seems reasonable to 
conclude that from the added memory krita uses while loading). By converting 
the image in the background, we won't need the imagemagick memory as long as 
we would when we would do a lazy conversion.

> Maybe we could even devise an algorithm to delete unused tiles from memory
> and cache them on disk; same with undo information.
Didn't we used to have a (not implemented) class that was supposed to do stuff 
like this? I think a tiles cache is the way we should do it. Convert 
everything to a format the datamanager can read efficiently, and dump parts 
of that on disk. As for the caching part itself: I think there are some nice 
algorithms for that (because they are extensively used in operating systems). 
The 'Least Recently Used' and 'Clock' algorithms come to my mind, but I don't 
know how well they'd fit in krita design.

On undo information: I recently found this screenshot of the gimp:
http://developer.gimp.org/screenshots/gimp-misc.png
Note the "Clear Undo History" dialog on the right, with an estimate of memory 
usage that will be freed. That would probably be a nice thing to have in 
krita as well (though keeping track of the size would be hard, I fear)

Bart Coppens