My ideas about GSOC Tiles project

[Boudewijn Rempt]

On Sunday 22 March 2009, Dmitry Kazakov wrote:
> Good evening, everybody!
>
> I would like to code Tiles GSOC project for krita. What should start with?
>
> I have already thought over some general ideas of the future project and
> wrote something like a "plan" for the project:
>
> 1) The first thing to implement is Mipmapping. I've come to a conclusion
> that mipmapping should be done at the level of Layer's projection (e.g.
> KisPaintLayer::Private {KisMipmappedPaintDevice projection; ...}). This
> KisMipmappedPaintDevice class should be derived from KisPaintDevice and
> support most of it's methods. The difference is that mipmapped class will
> have pyramid of scaled images.

Are you sure that KisPaintDevice is the right level, instead of 
KisDataMananager/KisTiledDataManager -- I'd have thought that adding a 
KisMipMappedDataManager would be a more likely solution. I was also thinking 
of havinga  KisUntiledDataManager for those paint devices that are ephemeral 
and small, like brush footprints.

> Why should it be put here? Let's look at the case when we have the only
> PaintLayer. In such a case, applying of any paintop we'll change 1-2 tiles.
> All the rest tiles will be untouched, and of course their scaled-down
> versions will stay untouched too. It turnes out that we have to scale only
> 1-2 tiles instead of scaling the whole image.

> The things are a bit more difficult with Adjustment Layers because most of
> them need the whole image for processing. But there are a few filters that
> could benefit from mipmapping at the layer stage. E.g. Perchannel Filter.
> It could process scaled-down picture while previewing (previewing is the
> _worst_ thing in this filter at the moment). All the filters could have a
> method KisFilter::NeedsOriginalImage() that would say whether filter can
> work on downscaled image.

Yes, such a flag is needed. We might want to do a little refactoring with all 
our flags, though. Maybe create an enum and a test function, following Qt4 api 
conventions.

> More than that, layer-level mipmapping can be used while creating the
> projection of the root layer.
>
> Question. Could someone tell me, will the result be different in these two
> situautions:
> Assume we have two *PaintLayers*.
> 1) first we Merge them, then Scale down
> 2) first we Scale them down, then Merge
> Will the result be different? (or, more precisely, will the result be
> *much* different)

I think we need to experiment with this. My gut feeling says "yes, there'll
be a discernable difference". Also note that the current zoom level of the 
canvas is rarely stable, people tend to zoom in and out a lot, which means 
that we don't win much by not compositing the levels that aren't in current 
use.

The gimp, btw, has currently already something like this, I believe. I haven't 
had time to check their source code yet, though.

> If the difference is too small, then we could use pyramid in
> KisMergeVisitor to create a projection.
>
> (the last issue can't be applied in case we have at least one Adjustment
> Filter in the stack, in this case we should consult with
> KisFilter::NeedsOriginalImage() first. BUT this approach works well with
> Filter Masks, as in such a case filters brake only one pyramid that can be
> quickly recalculated)
>
> 2) Locking and threading. I think locking should be made something like
> linux kernel's deferred irq subsystem. Every MipmappedPaintDevice should
> have its own working thread. Most of the time it sleeps on it's
> working_queue's semaphore. When work appears it goes to work_queue, the
> semaphore gets up and thread goes on calculating piramids for the tiles
> listed in workqueue.

Sounds good.

> In addition every tile should have it's own read/write semaphore.

Also think of COW and concurrent reads and so on. 

> 2b) Swapping. Again, one KisTiledDataManager - one swapper thread. This
> thread could work without workqueue. Just go though existing tiles and find
> unneeded. The concurrent access to tiles is controlled by tile's
> RW-semaphore.

Right now I thin, we have one swapper thread -- if it's a thread and not part
of the main loop -- because we have one big pool of tiles instead of separate 
pools for every paint device. This did give us quite a bit of performance 
boost back in the days when it was created this way.

> 3) In memory compression. I'm not sure i'll have time for good on-the-fly
> compression engine, but it can be made in swapper thread.

We have a good compression engine for on-the-fly work already -- Ariya Hidayat 
made it and did some experiments that showed that keeping lesser-used stuff in 
compressed form in memory before swapping (and uncompressing on undo) would 
indeed save us a lot of memory and also quite a bit of performance, since the 
compressed tile would take fewer memory reads to get into the processor cache.

> 4) Swapping to the PNG, TIFF
>
> Question. Could someone tell me, can tiff or png save to file only parts of
> the image? Or they need the whole image for compressing?

Cyrille should know about that.

> If the latter is true, what is the meaning of "pluggable png-tiff
> backends"?
>
> 5) Undo information. I've not thought about it properly. But is done at the
> level of tiles and won't disturb projections, i hope.

No -- but compression is also done at the tile level. But you can, if a tile 
gets mementoed, just compress the whole area, all levels, in one go.

> Well, that's all i've understood from the code and project ideas pattern at
> the wiki.

Pretty good work, actually!

> Could someone comment on this plan and tell me what should i do next?

It looks cool. Chew a bit on it and then enter a proposal with Google.

> PS:
> What does KisProjection do? Where is it used? I've not seen it's
> declarations in layers' classes.

It used to do multi-threading of recompositing and was owned by KisImage. I'm 
currently reworking it to work with the new stateless recompositing scheme, 
but I've run into a Threadweaver bug I haven't had time to fix yet.

-- 
Boudewijn Rempt | http://www.valdyas.org