KDE4 and video acquisition

Marco Lohse mlohse at cs.uni-sb.de
Wed Feb 1 08:32:15 GMT 2006

Laurent Pinchart wrote:
>>- leave as much of the hard work to the backends as possible :-P
> Speaking of backends, things are quite clear for sound (NMM, MAS, 
> GStreamer, ...), but what about video capture ? Are the required features 
> implemented in the backends we will use ? My goal is to have KDE4 support 
> webcams out of the box. I'm beginning to understand that it will require much 
> more work than I thought if I need to contact developers of all backends and 
> convince them that they need to change their API to expose required features. 
> The future doesn't currently look very bright to me :-(
>>>Audio/video sync will probably be one of the major problems. Even if the
>>>device can capture both audio and video, the two streams can be unlocked,
>>>and we must provide a way to resynchronize them.
>>Phonon does not really care. ;-) Well, what Phonon does is to tell the
>>backend what the developer wants. The backend then has to take care of
>>sync. For example I create an AvCapture object and set the AudioSource to
>>the soundcard and the VideoSource to a webcam. The VideoPath and the
>>AudioPath are both connected to an AvWriter object:
> [snip]
> I don't have enough experience with audio/video synchronisation to make 
> constructive comments, but don't underestimate the difficulty. For instance, 
> UVC webcams expose 2 clocks to the computer (internal device clock, used for 
> video capture, and USB bus clock), which must then use the information to 
> compute the jitter between the device clock and the computer internal clock. 
> The same operation is done for audio, and the audio and video must be 
> *processed* (this means modifying the audio and/or video rate by dropping 
> frames or changing the audio speed using digital filtering). You can put the 
> responsability of those operations on the backend, but make sure you 
> understand the issue before designing the Phonon API. Once again, it's a 
> double-edged knife. If the audio and video are not synchronized, developers 
> will blame Phonon.
> Sorry to sound so negative, but I think those points must be well understood, 
> otherwise Phonon will follow the path that arts took a few years ago.

NMM http://www.networkmultimedia.org does support synchronized capturing 
from different devices (e.g. a V4L(2) frame grabber/camera and a sound 
board). In fact, this feature was introduced in 2002: 

Cheers, Marco

More information about the kde-multimedia mailing list