[Kdenlive-devel] mlt jack issues - episode 3

[Ed Rogalsky]

Hi Dan,

the last days I worked on the jack seeking and I hope this is the last
major step before contributing the code to kdenlive.
There are some problems with the current jack_sync implementation.

I parsed the jack documantion and found that it is a little bit
unclear. Thats why I studied some jack clients like ardour or
qtractor and the jackd implementation. Here the results:

Jack Sync theory:
==============

    The jack_sync callback is designed only for position relocation
(seeking) and slow sync clients (sync callback registered).
    The idea is to give the clients enough time for seeking. Indeed
"video clients" are very slow in seeking.
    Starting jack transport is per defintion a "relocation". So
jack_sync is only called (for registered client) if a relocation is
    requested or the jack transport is started(ing). The jack_sync is
not the transport transition callback: started => stopped and so
    on. For getting the transport transition info it has to be polled
e.g. from the jack_process by calling the jack_transport_query
    function (follow_transport_slave principle). There can be fast
clients with no jack_sync callback registered.

    The jack transport is started if all clients return 1 from the
sync_callback or the registered timeout (here: 5s) elapse. After
    returning 1 the sync_callback is not called for the entire client.

Back to mlt:

Issue 1:      _last_pos property
======

The _last_pos property is not updated if we have the following timeline:

+---------+----------------------+
| blank    | video.mpg            |
+---------+----------------------+
6s            e.g. 20s

or

#sh: MLT_CONSUMER=sdl_preview ./melt -jack -blank 125 video-file.mpg

In this case you will notice that the jack transport is started after
5s when the timeout elapse and the jackd starts the transport.
The problem is that in the "blank" time the function

jackrack_get_audio

is not called and we don't get the position update.

jack_sync:

		else if ( position >= mlt_properties_get_position( properties,
"_last_pos" ) - 2 ) => based on the "_last_pos" this criteria is not
matched or incorrect
		{
			mlt_properties_set_int( properties, "_sync_guard", 0 );
			result = 1;
		}


case 1:

not updated "_last_pos" > position => timeout

case 2:

not updated "_last_pos" < position => jack transport starts, but hay
did we wait for mlt seeking???



Issue 2: mlt transport is started ignoring slower sync clients
======

jack_sync():
....
	else if ( state == JackTransportStarting )
	{
		result = 0;
		if ( !mlt_properties_get_int( properties, "_sync_guard" ) )
		{
			mlt_properties_set_int( properties, "_sync_guard", 1 );
			mlt_events_fire( properties, "jack-started", &position, NULL );
		}
		else if ( position >= mlt_properties_get_position( properties,
"_last_pos" ) - 2 )
		{
			mlt_properties_set_int( properties, "_sync_guard", 0 );
			result = 1;
		}
	}
...

Considering the fact "_sync_guard == 0" the "jack-started" event is
fired and the mlt transport starts. This is a problem if we have
some slower sync clients connected. The jackd holds the jack transport
in "starting" until all clients return 1 or the timeout elapse.
In this case the mlt client gets absolutely out of sync.

As mentioned above the mlt transport should be started from the
jack_process by polling the jack_transport_query (follow_slave
principle).
And in the jack_sync only "mlt seeking" should be requested. This is
the way it is implemented in ardour.

In my private mlt jack  branch I introduced the event "jack-starting"
- giving the appl the time for seeking or starting a very complex and
slow
enginge but not the transport :-))

See for inspiration:
https://github.com/eddrog/mlt/blob/jack/experimental/src/modules/jackrack/filter_jackrack.c

As a work around for "_last_pos" I'm syncing the jack position to
consumers position. Sorry for the "jack-last-pos-req" event. I tried
to reach the
consumers object from the filter. But as result I got just crashes.
In the appl I set the "_last_position" property with the conumers
position value.

My implementation is very experimental but I get pretty good sync
results (seeking while playing).


I know you have a better way to solve the issues but I think my way is
a good base for inspiration :-)))



Issue 3: sound starts to early about 1s
======

+---------+----------------------+
| blank    | video.mpg            |
+---------+----------------------+
6s            e.g. 20s

consumer=sdl_preview

We have again our problematic timeline. In this case the sound starts
to play ca. 1s to early. I tried to figure out why but hay.... :-))




Issue 4: sound plays after stopping (ca. 1s)
======

I have still the problem that after stopping you hear the sound
playing ca. 1s. It becomes very crazy if you have a transport loop
e.g. in ardour and
a sound track in kdenlive. After looping some times and stopping it
plays and plays and plays (ca. 5s or so). For me this issue is low
prio I think.




Ok now I'm done - puh.

regards

eddrog