[Kdenlive-devel] Fwd: mlt jack issues - episode 2

[Dan Dennedy]

On Sun, Aug 5, 2012 at 11:58 AM, Dan Dennedy <dan at dennedy.org> wrote:
> On Sun, Aug 5, 2012 at 7:02 AM, Ed Rogalsky <ed.rogalsky at gmail.com> wrote:
>> Hi Dan,
>>
>> the last weeks I thought my job will kill me :-)). But now I had some
>> time and here my new results:
>>
>> 1)  There is one issue with seeking with mlt jack if the consumer
>> (sdl_preview) is stopped or paused. The reason is the same as for our
>> ugly noise. The function in filter_jackrack.c:
>>
>> static int jackrack_get_audio( mlt_frame frame, void **buffer,
>> mlt_audio_format *format, int *frequency, int *channels, int *samples)
>> {
>> ...
>>
>>         if ( pos == mlt_properties_get_position( filter_properties,
>> "_jack_seek" ) )
>>         {
>>                 jack_client_t *jack_client =
>> mlt_properties_get_data(filter_properties, "jack_client", NULL );
>>                 jack_position_t jack_pos;
>>                 jack_transport_query( jack_client, &jack_pos );
>>                 double fps = mlt_profile_fps(
>> mlt_service_profile(MLT_FILTER_SERVICE(filter) ) );
>>                 jack_nframes_t jack_frame = jack_pos.frame_rate * pos / fps;
>>                 jack_transport_locate( jack_client, jack_frame );
>>                 mlt_properties_set_position( filter_properties,
>> "_jack_seek", -1 );
>>         }
>> ...
>> }
>>
>> is not called and the _jack_seek property is not evaluated. This means
>> that seeking on paused consumers is not possible. I think this is the
>> common use case - seeking is done on paused consumers.
>
> OK, but when you commence playback after a seek, then the other jack
> app seeks and plays. I did not it an important use case to require the
> other app to sync while paused. After all, we are paused, so there is
> no sound playing. But I guess this use case does help you locate an
> interesting area of the audio timelime by going to it in kdenlive.
>
>> Another big issue is resulting from this:
>> Seeking in Tracks without or muted audio is not possible at all. Very
>> bad for the following use case:
>>
>> Video is edited in kdenlive and audio in ardour/tractor. In Kdenlive I
>> don't have audio at all ==> jack seeking in kdenlive is not possible.
>
> In Shotcut, I do not have this problem because I attach the jackrack
> filter to the consumer and I tell the consumer audio_off=1. Then, you
> do not need to manually mute tracks.
>
>> In the function:
>>
>> static void on_jack_seek( mlt_properties owner, mlt_filter filter,
>> mlt_position *position )
>> {
>>         mlt_properties properties = MLT_FILTER_PROPERTIES( filter );
>>         mlt_log_verbose( MLT_FILTER_SERVICE(filter), "%s: %d\n",
>> __FUNCTION__, *position );
>>
>>         mlt_properties_set_int( properties, "_sync_guard", 1 );
>>         mlt_properties_set_position( properties, "_jack_seek", *position );
>>         return; <--- why
>
> I no longer remember. The following code was retained just in case I
> changed my mind about something.

Now, I remember that I did this to have tighter audio synchronization
between the jack clients when you are using the MLT jack audio output
as well. There can be workflows where this is important. This can be
best measured/witnessed by exporting the audio from a video project
and loading that into an audio editor. Then, start the mlt app with
jackrack filter and said video project and connect its jack ports to
system. Then, let both jack clients playout the same audio. This sync
problem is most noticeable when seeking while playing. However, in my
testing now, it (_jack_seek property) does not seem to be so much
better than removing the return statement, and it has the negative
side effect you are raising. Both approaches give good synchronization
when seeking while paused and then starting playback, and gives poor
sync when seeking while playing.

>>         mlt_profile profile = mlt_service_profile( MLT_FILTER_SERVICE(
>> filter ) );
>>         jack_client_t *jack_client = mlt_properties_get_data(
>> properties, "jack_client", NULL );
>>         jack_nframes_t jack_frame = jack_get_sample_rate( jack_client );
>>         jack_frame *= *position / mlt_profile_fps( profile );
>>
>>         jack_transport_locate( jack_client, jack_frame );
>> }
>>
>> you return after setting the property "_jack_seek" without calling the
>> jack_transport_locate( jack_client, jack_frame ); . I don't understand
>> that the sync is not done here. This is done later in the function
>> "jackrack_get_audio". But this function is not called in cases mentioned above.
>>
>> In my private mlt branch I removed the return and the setting of the
>> "_jack_seek" property it worked pretty good so far.
>>
>> Have you got an idea how to solve this issues???
>
> I will experiment with that change and get back to you.
>
>>
>> 2)  The crazy noise is now fixed. But on stopping/pausing jack
>> transport it takes at least one second until sound stops to play. I
>> tried this with shotcut with the same result. In ardour the sound
>> stops immediately after pressing the stop key.Is there a way to
>> optimize this behaviour? (of course low prio).
>>
>
> no comment at this time
>
>>
>> 3)  By the way. I played a little bit with shotcut. On my computer I
>> have the same behaviour with opengl and jack (I reported this some
>> time ago with kdenlive). If shotcut is connected to jack on exit it hangs
>> forever. For this I want to try jack1 and perhaps another linux installation.
>
> On my system, I can reproduce this in Shotcut when not using OpenGL. I
> will look into it.
>
>>
>> PS: With the hack in mlt (removing return ...) and the jack enabled
>> kdenlive I made a 15min film. The video was done in kdenlive and the
>> audio in ardour. It worked for me so far. I think now I am a step
>> closer to the end :-)))
>
> good job!
>
>> regards
>> eddrog
>>