KDE4 and video acquisition
mlohse at cs.uni-sb.de
Wed Feb 1 08:32:15 GMT 2006
Laurent Pinchart wrote:
>>- leave as much of the hard work to the backends as possible :-P
> Speaking of backends, things are quite clear for sound (NMM, MAS,
> GStreamer, ...), but what about video capture ? Are the required features
> implemented in the backends we will use ? My goal is to have KDE4 support
> webcams out of the box. I'm beginning to understand that it will require much
> more work than I thought if I need to contact developers of all backends and
> convince them that they need to change their API to expose required features.
> The future doesn't currently look very bright to me :-(
>>>Audio/video sync will probably be one of the major problems. Even if the
>>>device can capture both audio and video, the two streams can be unlocked,
>>>and we must provide a way to resynchronize them.
>>Phonon does not really care. ;-) Well, what Phonon does is to tell the
>>backend what the developer wants. The backend then has to take care of
>>sync. For example I create an AvCapture object and set the AudioSource to
>>the soundcard and the VideoSource to a webcam. The VideoPath and the
>>AudioPath are both connected to an AvWriter object:
> I don't have enough experience with audio/video synchronisation to make
> constructive comments, but don't underestimate the difficulty. For instance,
> UVC webcams expose 2 clocks to the computer (internal device clock, used for
> video capture, and USB bus clock), which must then use the information to
> compute the jitter between the device clock and the computer internal clock.
> The same operation is done for audio, and the audio and video must be
> *processed* (this means modifying the audio and/or video rate by dropping
> frames or changing the audio speed using digital filtering). You can put the
> responsability of those operations on the backend, but make sure you
> understand the issue before designing the Phonon API. Once again, it's a
> double-edged knife. If the audio and video are not synchronized, developers
> will blame Phonon.
> Sorry to sound so negative, but I think those points must be well understood,
> otherwise Phonon will follow the path that arts took a few years ago.
NMM http://www.networkmultimedia.org does support synchronized capturing
from different devices (e.g. a V4L(2) frame grabber/camera and a sound
board). In fact, this feature was introduced in 2002:
More information about the kde-multimedia