[Kdenlive-devel] Color spaces and MLT

[Dan Dennedy]

On Wed, Jul 21, 2010 at 10:51 AM, Simon Eugster <simon.eu at gmail.com> wrote:
> Dear friends,
>
> I've just added an RGB parade to kdenlive. While testing I found
> something interesting: Although I'm painting on a rect with height 256
> (as atm we're dealing with 8 bits per channel only) and scaling the
> parade afterwards to the target height, I got kind of scanlines.
> Compare these two images:
> http://granjow.net/uploads/kdenlive/kdenlive-mlt-rgbparade-noeffect.png
> http://granjow.net/uploads/kdenlive/kdenlive-mlt-rgbparade-witheffect.png
> With the MLT effect there are scanlines, without it there are none.
>
> This is quite interesting, as this visualizes how color information
> gets lost when converting between different color spaces (according to
> [1], MLT is using YUV422, the input image is RGB).

It does not operate exclusively in YUV422. The color/image-format
conversion implementation depends on your build and your version of
libswscale (>= 0.7.2) whether MLT uses libswscale or inbuilt routines.
Maybe the problem is in the composite transition.

> Basically I just wanted to show you this because it is interesting.
> But in the long term one of our goals should also be to avoid either
> conversion to other color spaces or the loss of information caused by
> the conversion.

This already exists, but some services only operate in specific
colorspace/format. Last year, I overhauled the conversion handling in
MLT. Services can pass through the colorspace/format that was
requested of it or it can explicitly state which colorspace/format it
requires. Then, a combination of the framework and some core services
will take care of converting a frame's image only as-needed in the
sequence of image processing steps.

For example, if you load a still image, and apply a RGB-based frei0r
filter, a view it paused in Kdenlive, then it should remain RGB
throughout. However, if you press play, then the SDL consumer does a
YUV output, and the conversion takes place after the frei0r filter. Of
course, it is a bit more complicated than this in practice because
there are "automatic" normalization filters for deinterlace, scaling,
and padding, each of which may have a limitation. Currently, among
those, only the deinterlace filter is limited to YUV. Furthering the
example, if you use a YUV video source, then when paused the RGB
conversion takes place immediately prior to frei0r. When playing, it
must do RGB conversion prior to frei0r and back to YUV after the
frei0r filter. Add more frei0r filters, and it remains RGB from one
filter to the next. Put a MLT YUV-based filter (everything except
frei0r, affine, boxblur, and burningtv) between frei0r filters, press
play, and it will do YUV->RGB->YUV->RGB->YUV - not ideal, but what the
user asked for. Obviously, it would be a bad thing if the framework
attempted to reduce conversions by automatically reordering filters.

-- 
+-DRD-+