More on threading

[Boudewijn Rempt]

A follow-up to my last mail thinking about what went wrong with threading
in 2.0, this is just a start for getting to a design for threading in Krita.
(And doing it right, this time).

I'll also add these documents to krita/docs

--- places to use threading ---

Threading can be useful in the following places (this is meant to be a
totally exhausting list, and a list of places where it is smart to use
threading: that is a per-item decision -- if I've forgotten, please
add to it ):

 A) handle all layers or a subset of layers concurrently
   * transforming layers in an image
   * converting the colorspace of layers in an image

 B) chunked processing of a single paintdevice
   * filtering a paintdevice
   * converting the colorspace of a paintdevice
   * compositing the projection of (part of) a node stack
   * running a generator on a paintdevice
   * separating channels in a paintdevice
   * computing the histogram of a paintdevice
   * copying pixel data from one paintdevice to another
   * transforming a single paintdevice (mirror, scale, skew, rotate)

 C) encapsulating an action in a thread that processes in the
background, to keep the user interface responsive
   * printing
   * compressing tile data
   * writing out tile data to the swapfile
   * exporting a file
   * saving a file

 D) queueing gui input events
   * painting with the freehand tool

 E) preparing and loading resources so the application is quickly
ready for the user
   * KoResources (brushes, patterns, paintop presets, gradients
   * plugins
   * previews in the filter gallery

 F) preparing the projection for the redisplay. This contains the
following steps:
   * converting from a ready projection to a color-correct QImage
   * scaling that QImage to the right size
   * pasting it in the scaled QImage cache
   * notifying the canvas it can update itself.


--- available threading api's ---


There are the following options for implementing threading in Krita:

 * ThreadWeaver
    * KoAction
    * KisThreadedApplicator
 * QThread
 * QThreadPool and QRunnable
 * QtConcurrent: QFuture and QFutureWatcher 
 * KoProgressUpdater (should add method to check whether cancel() has
been called)

--- considerations ---


There are some considerations necessary when designing for threads:

 * state: a threaded action should have all the state necessary to
finish the action: this state should not be changed when running the
thread.

 * progress reporting: the progress report should report the total of
all subtasks.

 * cancelling: the cancel widget of the progress reporting widget
needs to be connected to all subthreads somehow. We need to undo all
the work the subthreads have done. Quitting krita while threads are
running is a subproblem of this.

 * undo action adding: when all subthreads that make up a single
action are done, we need to add the action to the undo stack.

 * locking of resources (such as nodes): when an action, whether
chunked or not, is active on a node, the node should be locked for
access by new actions: new actions should be queued.

 * memory handling: threading should be fire and forget, there should
not be a necessity to delete objects after the thread is done from the
code that started the thread

 * In the case of recomposition, we have the problem that some filters
cannot be run chunked. There are two cases: 

 * adjustment layers: extra If we encounter an
adjustment layer with a filter that cannot be chunked we need to wait
until all the chunks in the job in the current group are done, apply
the adjustment layer in one chunk, then continue from there chunked.

 * filter masks: if we encounter a filter mask with an
unchunkable filter, we need apply the filter to the source layer in
one go, mark everything as clean, and then recomposite from there in
chunks again.
       

--- notes ---

 * Progress reporting: use KoProgressUpdater (or QFuture and QFutureWatcher?)
 * chunked processing: use KisThreadedApplicator (based on ThreadWeaver)
 * in the tile system: use QReadWriteLock
 * threading shoud be fire and forget
 * For queuing of actions on locked nodes: KoAction
 * For simultaneous work on multiple layers: a new class
 * KisThreadedApplicator is fire and forget: calling execute queues
the subtasks and returns, should not wait. (It does now, but that's wrong)