[Kdenlive-devel] [Mlt-devel] mlt jack issues - episode 2

[Ed Rogalsky]

>>> Hi Dan,
>>>
>>> the last weeks I thought my job will kill me :-)). But now I had some
>>> time and here my new results:
>>>
>>> 1)  There is one issue with seeking with mlt jack if the consumer
>>> (sdl_preview) is stopped or paused. The reason is the same as for our
>>> ugly noise. The function in filter_jackrack.c:
>>>
>>> static int jackrack_get_audio( mlt_frame frame, void **buffer,
>>> mlt_audio_format *format, int *frequency, int *channels, int *samples)
>>> {
>>> ...
>>>
>>>         if ( pos == mlt_properties_get_position( filter_properties,
>>> "_jack_seek" ) )
>>>         {
>>>                 jack_client_t *jack_client =
>>> mlt_properties_get_data(filter_properties, "jack_client", NULL );
>>>                 jack_position_t jack_pos;
>>>                 jack_transport_query( jack_client, &jack_pos );
>>>                 double fps = mlt_profile_fps(
>>> mlt_service_profile(MLT_FILTER_SERVICE(filter) ) );
>>>                 jack_nframes_t jack_frame = jack_pos.frame_rate * pos / fps;
>>>                 jack_transport_locate( jack_client, jack_frame );
>>>                 mlt_properties_set_position( filter_properties,
>>> "_jack_seek", -1 );
>>>         }
>>> ...
>>> }
>>>
>>> is not called and the _jack_seek property is not evaluated. This means
>>> that seeking on paused consumers is not possible. I think this is the
>>> common use case - seeking is done on paused consumers.
>>
>> OK, but when you commence playback after a seek, then the other jack
>> app seeks and plays. I did not it an important use case to require the
>> other app to sync while paused. After all, we are paused, so there is
>> no sound playing. But I guess this use case does help you locate an
>> interesting area of the audio timelime by going to it in kdenlive.

Exactly. Kdenlive or other jack enabled app that provides the
appropriate info.


>>> Another big issue is resulting from this:
>>> Seeking in Tracks without or muted audio is not possible at all. Very
>>> bad for the following use case:
>>>
>>> Video is edited in kdenlive and audio in ardour/tractor. In Kdenlive I
>>> don't have audio at all ==> jack seeking in kdenlive is not possible.
>>
>> In Shotcut, I do not have this problem because I attach the jackrack
>> filter to the consumer and I tell the consumer audio_off=1. Then, you
>> do not need to manually mute tracks.

I do it in the same way (audio_off=1). But what I mean is:

1) I just have video data without audio

or

2) In kdenlive I mute the embedded audio in a video track because I
record additionaly the audio that I import directly to ardour/tractor.

In this case the function jackrack_get_audio is not called too and
the _jack_seek property is not evaluated => jack_locate is not
called and the other jack clients are not synced at all. Also
while jack transport is running.
But I think this problem is similar to that mentioned above.
Removing "return" solves it anyways.


>>> In the function:
>>>
>>> static void on_jack_seek( mlt_properties owner, mlt_filter filter,
>>> mlt_position *position )
>>> {
>>>         mlt_properties properties = MLT_FILTER_PROPERTIES( filter );
>>>         mlt_log_verbose( MLT_FILTER_SERVICE(filter), "%s: %d\n",
>>> __FUNCTION__, *position );
>>>
>>>         mlt_properties_set_int( properties, "_sync_guard", 1 );
>>>         mlt_properties_set_position( properties, "_jack_seek", *position );
>>>         return; <--- why
>>
>> I no longer remember. The following code was retained just in case I
>> changed my mind about something.
>
> Now, I remember that I did this to have tighter audio synchronization
> between the jack clients when you are using the MLT jack audio output
> as well. There can be workflows where this is important. This can be
> best measured/witnessed by exporting the audio from a video project
> and loading that into an audio editor. Then, start the mlt app with
> jackrack filter and said video project and connect its jack ports to
> system. Then, let both jack clients playout the same audio. This sync
> problem is most noticeable when seeking while playing. However, in my
> testing now, it (_jack_seek property) does not seem to be so much
> better than removing the return statement, and it has the negative
> side effect you are raising. Both approaches give good synchronization
> when seeking while paused and then starting playback, and gives poor
> sync when seeking while playing.

I think I reproduced this in mixed mode:

1) Video and Audio in kdenlive.
2) The same audio in ardour.
3) Jack enabled seeking => could be better.

I think we will find a solution if we are closer to production use. For me
seeking while paused is enought for now.

>>>         mlt_profile profile = mlt_service_profile( MLT_FILTER_SERVICE(
>>> filter ) );
>>>         jack_client_t *jack_client = mlt_properties_get_data(
>>> properties, "jack_client", NULL );
>>>         jack_nframes_t jack_frame = jack_get_sample_rate( jack_client );
>>>         jack_frame *= *position / mlt_profile_fps( profile );
>>>
>>>         jack_transport_locate( jack_client, jack_frame );
>>> }
>>>
>>> you return after setting the property "_jack_seek" without calling the
>>> jack_transport_locate( jack_client, jack_frame ); . I don't understand
>>> that the sync is not done here. This is done later in the function
>>> "jackrack_get_audio". But this function is not called in cases mentioned above.
>>>
>>> In my private mlt branch I removed the return and the setting of the
>>> "_jack_seek" property it worked pretty good so far.
>>>
>>> Have you got an idea how to solve this issues???
>>
>> I will experiment with that change and get back to you.
>>
>>>
>>> 2)  The crazy noise is now fixed. But on stopping/pausing jack
>>> transport it takes at least one second until sound stops to play. I
>>> tried this with shotcut with the same result. In ardour the sound
>>> stops immediately after pressing the stop key.Is there a way to
>>> optimize this behaviour? (of course low prio).
>>>
>>
>> no comment at this time
>>
>>>
>>> 3)  By the way. I played a little bit with shotcut. On my computer I
>>> have the same behaviour with opengl and jack (I reported this some
>>> time ago with kdenlive). If shotcut is connected to jack on exit it hangs
>>> forever. For this I want to try jack1 and perhaps another linux installation.
>>
>> On my system, I can reproduce this in Shotcut when not using OpenGL. I
>> will look into it.

This is strange. Just the opposite. I think there is something in the hood.

>>>
>>> PS: With the hack in mlt (removing return ...) and the jack enabled
>>> kdenlive I made a 15min film. The video was done in kdenlive and the
>>> audio in ardour. It worked for me so far. I think now I am a step
>>> closer to the end :-)))
>>
>> good job!

Thanx

>>> regards
>>> eddrog
>>>

PS: If you select in Shotcut jack as input you see the port named as "Shotcut"
- how is it possible?