[Kdenlive-devel] Color spaces and MLT

[Simon Eugster]

2010/7/21 Dan Dennedy <dan at dennedy.org>:
> On Wed, Jul 21, 2010 at 10:51 AM, Simon Eugster <simon.eu at gmail.com> wrote:
>> Dear friends,
>>
>> I've just added an RGB parade to kdenlive. While testing I found
>> something interesting: Although I'm painting on a rect with height 256
>> (as atm we're dealing with 8 bits per channel only) and scaling the
>> parade afterwards to the target height, I got kind of scanlines.
>> Compare these two images:
>> http://granjow.net/uploads/kdenlive/kdenlive-mlt-rgbparade-noeffect.png
>> http://granjow.net/uploads/kdenlive/kdenlive-mlt-rgbparade-witheffect.png
>> With the MLT effect there are scanlines, without it there are none.
>>
>> This is quite interesting, as this visualizes how color information
>> gets lost when converting between different color spaces (according to
>> [1], MLT is using YUV422, the input image is RGB).
>
> It does not operate exclusively in YUV422. The color/image-format
> conversion implementation depends on your build and your version of
> libswscale (>= 0.7.2) whether MLT uses libswscale or inbuilt routines.
> Maybe the problem is in the composite transition.

libswscale is at 0..6 here in debian sid.

>> Basically I just wanted to show you this because it is interesting.
>> But in the long term one of our goals should also be to avoid either
>> conversion to other color spaces or the loss of information caused by
>> the conversion.
>
> This already exists, but some services only operate in specific
> colorspace/format. Last year, I overhauled the conversion handling in
> MLT. Services can pass through the colorspace/format that was
> requested of it or it can explicitly state which colorspace/format it
> requires. Then, a combination of the framework and some core services
> will take care of converting a frame's image only as-needed in the
> sequence of image processing steps.
>
> For example, if you load a still image, and apply a RGB-based frei0r
> filter, a view it paused in Kdenlive, then it should remain RGB
> throughout. However, if you press play, then the SDL consumer does a
> YUV output, and the conversion takes place after the frei0r filter. Of
> course, it is a bit more complicated than this in practice because
> there are "automatic" normalization filters for deinterlace, scaling,
> and padding, each of which may have a limitation. Currently, among
> those, only the deinterlace filter is limited to YUV. Furthering the
> example, if you use a YUV video source, then when paused the RGB
> conversion takes place immediately prior to frei0r. When playing, it
> must do RGB conversion prior to frei0r and back to YUV after the
> frei0r filter. Add more frei0r filters, and it remains RGB from one
> filter to the next. Put a MLT YUV-based filter (everything except
> frei0r, affine, boxblur, and burningtv) between frei0r filters, press
> play, and it will do YUV->RGB->YUV->RGB->YUV - not ideal, but what the
> user asked for. Obviously, it would be a bad thing if the framework
> attempted to reduce conversions by automatically reordering filters.

Thank you, this was very interesting reading.

Simon