High level capture API state of development

[Matthias Kretz]

Hi,

On Sunday 03 August 2008 13:00:55 umberleigh wrote:
> I'm curious about the state of development of the high level capture
> API

I did some experiments and some work that will surely be usable. The main 
problem for it to go forward is missing decisions:

How should the API look like:

1. add a MediaSource(AudioCaptureDevice) ctor and let MediaObject be the API 
for doing capture
Problems:
1.a. what about all the parts of MediaObject that don't apply to capture?
1.b. how is the device actually opened? i.e. if you want real-time, high-
quality capture you need to be able to adjust and know many low-level things. 
The approach could be to only use AudioDataCapture for it and let it figure 
out the best capture it can do from what the program requires.

2. add AvCapture object that is similar to MediaObject but provides only the 
few function necessary for capture (select device, start/stop)

3. go all the way and add a "Graph"/"Synchronizer"/"Controller" class which 
define what sources (and sinks) need to be controlled in sync and is more 
flexible in the number of sources. There you could just add multiple hardware 
devices and request to capture from them in sync (this is a hard problem to 
get right (impossible?) unless this is explicitly restricted to hardware with 
a shared clock).

How should it be implemented:

1. the PCM capture code would very likely be the same for all backends on the 
same operating system - i.e. the capture implementation should not depend on 
the media framework used. Therefore it could all be implemented right in 
libphonon with plugins for the different low-level backends (ALSA/OSS/Pulse)
Problems:
1.a. it's hard or impossible to integrate this functionality with the 
functionality of the backends
1.b. Pulse is going to have the first sane PCM capture API on Linux, this is 
how Phonon should also allow access. Now how do you make that API work with 
ALSA and OSS properly?

1'. to solve 1.a. the implementation could be in a library that Phonon 
backends link to. Then it should be possible to integrate the functionality 
much better.

2. implement capture in the backends and specific to the backends

BTW: AFAIK gstreamer does not fulfil the requirements for a low-latency, real-
time audio API. Hmm, badly worded. You can do low-lat, real-time audio with 
gst. The API just doesn't allow the lowest-latency-ever-possible-on-your-
machine. So, I'd be glad for hard numbers on this topic. If we can simply let 
the gst backend use gst for all capture functionality, it should be less work 
(but then otoh, if we don't write the other code we can dump all other 
backends on Linux - which is allowed to happen some day, :-) I just don't know 
when the time is right, yet).

> and whether there's anything I could do to help?

You decide. :-) This stuff sadly isn't easy (it was until Trolltech said they 
want Phonon to be able to become a multimedia API that can do everything :-P )

But anyway, first we need to decide this stuff above. Then perhaps we can 
flesh out some tasks that you can help with. You're also welcome to share your 
thoughts on the above.

> I'm an
> inexperienced programmer but I'm keen to learn and it would 'scratch
> an itch' so to speak, as I want to write a program to capture audio
> and i'm unhappy with existing applications to do this which either
> don't work, rely on OSS, or are cumbersome to setup and use.

-- 
________________________________________________________
Matthias Kretz (Germany)                            <><
http://Vir.homelinux.org/
MatthiasKretz at gmx.net, kretz at kde.org,
Matthias.Kretz at urz.uni-heidelberg.de