[Kdenlive-devel] [Mlt-devel] non-25 fps broken

[Dan Dennedy]

On Sunday 28 October 2007, Dan Dennedy wrote:
> Hey, as a result of this bug:
> http://www.kdenlive.org/mantis/view.php?id=23
> I have determined that MLT and therefore kdenlive is currently broken for
> any profile not using 25 fps. By broken, I mean various things related to
> editing, particularly in and out points are not working as expected. This
> occurred with the introduction of mlt_profile. I have in my MLT working
> copy a bugfix that I am testing.

I want to document my findings during the analysis.

The framework maintains a "fps" property on all producers including tractor, 
multitrack, and playlist, which are derived from producer. This makes "fps" 
rather stateful with multiple instances of that state. A feature of 
mlt_profile is to eliminate requirement of an environment variable 
(MLT_NORMALISATION) as a means of passing parameters. It was a kludge for an 
API defficiency. The profile property of mlt_consumer is intended to replace 
it as a convenience over having to continually specifying all the profile's 
properties.

Due to order of initialization, some objects are initialized with default 
values, obviously, with the default equivalent to profile dv_pal. Typically, 
the consumer is constructed last and connected to the last producer. 
Therefore, with fps being stateful and the consumer profile property applied 
in a late manner, some timings were based on the initial, default PAL fps 
value of 25. Therefore, even with a NTSC-based consumer profile, the edit 
points were interpreted with the wrong fps.

The framework does not really use the producer fps property much outside 
setting it and propogating it (onto the frame and other producers). The 
modules do not use it much either. Moreover, they _never_ set it. In fact, 
there is an unofficial policy that a module's producer must conform to the 
fps or fail initialization because there is no core module filter to 
reconcile frame rate differences (between producers or between producer and 
consumer). It just so happens producer_avformat is adaptive to any frame 
rate.

To fix the bug, I made everything accessing the fps producer property use the 
existing mlt_producer_get_fps() instead. Then, I changed 
mlt_producer_get_fps() to simply return mlt_profile_fps(). However, really it 
should reference the final consumer's fps property as, again, the profile is 
merely a convenience to set the consumer property and can be overridden. As a 
resolution to the redundant fps property, I totally removed it from the 
producer side of the framework and all related modules. Left in its place is 
a bunch of calls to mlt_producer_get_fps().

Perhaps some day we should require every producer to set its fps and then 
allow a filter to reconcile differences. Similar to how fezzik "normalises" 
resolution and audio sample rate, it could automatically add a framerate 
normalisation filter. The core module could have a duplicate/drop algorithm 
and motion_est could provide one as well. In some cases, 
mlt_producer_get_fps() calls should be replaced with a mlt_consumer_get_fps( 
mlt_service_get_last_consumer() ).

However, this bug also brings to light an issue if I change the consumer's fps 
after editing. For example, I may want a low bitrate encoding with a lower 
frame rate. Since all edit decisions are expressed in frames, they are no 
longer valid at the lower frame rate.

All of this was not a major issue for the original requirements (ITU 601 NTSC 
or PAL for SD broadcast) since each installation tended to pick one of the 
two options and stick with it for all inputs and outputs.