Processing framework and color space conversion

[Stefano Bonicatti]

Hello again :)
2015-01-25 10:35 GMT+01:00 Dmitry Kazakov <dimula73 at gmail.com>:

> Hi, Stefano!
>
> Short answer:
>
>    - If I were you I would take the following plan:
>
>
>    1. Port image color space conversion to the strokes framework as it is
>    now. That is one layer --- one thread provided by the framework.
>    2. Then talk to the painters and ask them to test. Given that most of
>    the painters have i5 (4 cores) and work with images that have more than 4
>    layers, it is highly probable (thought should be measured, of course) that
>    no further steps will be needed. If measurements show that we still
>    underutilize CPU power (using VTune for example), we can always implement
>    further optimizations using QtConcurrent framework (we already use it in
>    KisAutoBrush). Or we can improve KisProcessingApplicator class to handle
>    that case if we end-up over-utilize CPU. All such optimization work should
>    be always controlled by very tight measurements. Otherwise we can end up
>    spending a month of work on making it 5-10% faster :)
>
>
>    - Yes, you were right about the two-stage openGL update. I almost
>    forgot about out talk. It should be done like resizing of the image, so
>    KisCanvas2 should handle colorSpaceChanged() signal.
>
> Makes sense to me.

KisUpdatesScheduler is much higher level of abstraction than a "generic
> scheduler". It should know quite a lot about how the image should be
> processed and how jobs interfere each. The thing you are talking about is
> already implemented in Qt, it is called QtConcurrent framework. And you can
> use it. Though there are risks that the CPU will be saturated. But we have
> it in KisAutoBrush and it seems to work quite fine.
>

Sorry i think i wrote my sentence wrongly, what i meant was that C# TPL and
TBB has a generic scheduler, we have a scheduler and it is NOT generic (so
we agree on that :P).
Though as you were concerned about using too many threads i'm too, that's
why i was asking if KisUpdateScheduler can do something similar to what TBB
or TPL does, because my point was to use the threads KisUpdateScheduler
already controls, so that only if it is really needed, a new thread is
spawned.

Think of a thread stuck in I/O or the CPU is underutilized even if there
are 4 KisUpdateScheduler threads (and the CPU has 4 cores), and a new job
is enqueued, that job would have to wait that one of the threads finish its
work, not knowing if that "work" is waiting on I/O or underutilizing CPU.
Would be better to be able to check that case and spawn a new thread, even
if it exceeds the threads per core number, to use the CPU more.

Now you suggest about using QtConcurrent framework, but with my quick
search i'm not able to find much information on how it handles the
underlying threads (probably it's better that i'll look into the code), and
as it says, i guess it just has a thread pool as in "the old days". That's
not in any way comparable to what TBB or TPL does :P.
Those deals with the concept of tasks/jobs (more like KisUpdateScheduler
does), but they tightly controls the threads they spawn so that workload is
equally shared among those (and not only that.. you can write data parallel
code, task parallel code, pipelines etc).

Also, stacking different "frameworks" that uses their threads in different
manner, plus having a couple of static threads that are rarely used might
not be the best idea.

Obviously "fixing" this is easier to be said than done, though since we
have stroke jobs that can be concurrent, "partially concurrent", blocking
etc, it can be a good reason to have a system like that, so not "blindly"
hard capping threads numbers, but doing something more sophisticated.

Hope i didn't missed anything in the code that prove me quite wrong, but if
so please bash me :P.

Now i've gone offtopic enough, just wanted to give a heads up about a
possible future improvement, i'll deal with what we have now, thanks for
all ^^.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kimageshop/attachments/20150125/93c9ffe4/attachment-0001.html>